Top 10 system design interview questions

Go step by step, saying what bottleneck each step solves:

1. One server with the app and database. Fine for thousands of users.
2. Separate the database onto its own machine; add indexes and fix slow queries.
3. Stateless app servers behind a load balancer: sessions in Redis or tokens, uploads in object storage, so you can add servers freely.
4. Caching: Redis for hot reads, a CDN for static assets and cacheable pages.
5. Database read replicas for read-heavy traffic; connection pooling.
6. Asynchronous work through queues and workers (emails, reports, image processing).
7. Sharding or splitting databases by domain when writes outgrow one primary.
8. Split services where teams or scaling needs diverge; add monitoring, autoscaling and multi-zone redundancy throughout.

The key message: scale based on measured bottlenecks, not up front.

Caching keeps frequently read data in fast storage (memory) to cut latency and database load.

• Levels: browser, CDN, application memory, distributed cache (Redis/Memcached), database buffer.
• Strategies: cache-aside (app reads cache, falls back to DB, fills cache: most common), read-through, write-through (write to both, always fresh), write-back (write to cache, persist later: fast but risky).
• Eviction: LRU, LFU, TTL.

• Problems:

  • Stale data → invalidate on write, short TTLs.
  • Cache stampede (many requests miss at once) → request coalescing/locks, early refresh.
  • Hot keys → replicate the key or add a local in-process cache.
  • Cache penetration (lookups for keys that don't exist) → cache negative results, bloom filters.

In a distributed data store, when a network partition splits nodes, you can't have both:

• Consistency: every read returns the most recent write, or an error.
• Availability: every request receives a non-error response, which may be stale.

Partition tolerance isn't optional in real networks, so the choice is effectively C or A during a partition. A bank ledger or inventory count prefers consistency (reject or wait rather than show wrong data). A social feed, like counts or a product catalogue prefer availability (stale for a few seconds is fine).

Mention PACELC for extra credit: even without partitions, there's a trade-off between latency and consistency.

A load balancer spreads incoming requests across servers to improve throughput and availability. It removes unhealthy servers using health checks, and often terminates TLS.

Algorithms: round robin (in turn), weighted round robin (bigger servers get more), least connections (best for uneven request lengths), IP hash / consistent hashing (the same client goes to the same server; useful for caches or sticky sessions).

Layer 4 balancers route TCP connections (fast, protocol-agnostic); Layer 7 balancers understand HTTP and can route by path, host or header (/api to one pool, /static to another). The load balancer itself must be redundant (an active-passive pair or a managed cloud LB).

Decide from data shape, access patterns and consistency needs:

• SQL (PostgreSQL/MySQL): related data, joins, ad-hoc queries, transactions, strong consistency. Orders, payments, users, applications.
• Key-value (Redis, DynamoDB): simple lookups by key at huge scale; sessions, carts, counters.
• Document (MongoDB): flexible, nested records read as a whole; content and catalogues with varying attributes.
• Wide-column (Cassandra): very high write throughput, time series, known query patterns.
• Search (Elasticsearch): full-text and faceted search.
• Graph (Neo4j): relationship-heavy queries (recommendations, fraud rings).

Real systems mix them (polyglot persistence). Starting with a relational database and adding specialised stores for specific needs is a defensible default.

Replication copies data from a primary to replicas. It scales reads and provides failover. Asynchronous replication is fast but replicas can lag (a user may not see their own update immediately); synchronous replication is consistent but slower.

Sharding splits data across multiple databases by a shard key (user id, tenant id, region). It scales writes and storage. Challenges: choosing a key that spreads load evenly and keeps related data together, cross-shard queries and transactions, rebalancing (consistent hashing helps), and hot shards (a celebrity user).

Order of operations: optimise queries and indexes → cache → replicas → partitioning → sharding.

A message queue lets a producer hand work to consumers asynchronously.

Benefits: decoupling (services don't need to be up at the same time), absorbing spikes (the queue buffers bursts; workers process at their own pace), reliability (retries, dead-letter queues), scalability (add workers).

Examples: sending emails and notifications after an order, video transcoding, generating PDFs, processing payments webhooks, updating search indexes, streaming click events into analytics (Kafka).

Points to mention: at-least-once delivery means duplicates are possible, so make consumers idempotent; ordering is guaranteed only within a partition (Kafka) or not at all (most queues); monitor queue length and age.

Requirements: limit each client (user id, API key or IP) to N requests per window; low latency; works across many servers.

Algorithms:

• Fixed window counter: count per minute; simple, but allows bursts at window edges.
• Sliding window log / counter: smoother and more accurate.
• Token bucket: tokens refill at a steady rate; each request spends one; allows short bursts up to the bucket size. Most widely used.
• Leaky bucket: processes at a constant rate; smooths traffic.

Implementation: keep counters in Redis with atomic operations (INCR with expiry, or a Lua script for a token bucket) so all app servers share the state. Return 429 Too Many Requests with Retry-After and rate-limit headers. Place it at the API gateway or as middleware. Discuss what happens if Redis is down (fail open for most APIs, fail closed for login and OTP).

Clarify: shorten, redirect, custom alias, expiry, analytics; 100M redirects/day; read-heavy (~100:1).

API: POST /links {url} → {code}; GET /{code} → redirect.

Code generation: base62 of a unique id (from a counter service or Snowflake-style generator) gives short, collision-free codes; 7 characters ≈ 3.5 trillion codes. Hashing the URL (MD5 → first 7 chars) needs collision handling.

Storage: code → url, owner, created_at, expires_at. Lookups are by key, so a relational table with the code as primary key or a key-value store both work.

Read path: load balancer → stateless service → Redis cache → database. Use 302 if you need click counts (browsers cache 301). Log clicks asynchronously via a queue.

Scale and reliability: replicas or a distributed key-value store, CDN edge caching for hot links, rate-limit creation, and scan for malicious URLs.

Monolith: one deployable unit. Simple to build, test, deploy and debug; function calls instead of network calls; one database with easy transactions. It can become hard to change when large teams work on it.

Microservices: independent services per business capability, each with its own data. Independent deployment and scaling, team autonomy, fault isolation, technology freedom. The costs: network latency and failures, distributed transactions (sagas), data consistency, much harder debugging (needs tracing), and DevOps overhead.

A good answer: start with a modular monolith with clear boundaries; extract a service when there's a concrete reason: a part with very different scaling needs, a separate team, or different reliability requirements. Microservices solve organisational scaling problems more than technical ones.