Building blocks: load balancers, caches, CDNs and queues

Load balancer

Spreads traffic across several stateless app servers, removes unhealthy ones (health checks) and terminates TLS.

• Algorithms: round robin, least connections, IP hash (sticky sessions).
• Layer 4 (TCP, fast) vs Layer 7 (HTTP, can route by path or header).
• For this to work, servers must be stateless: keep sessions in Redis or use tokens.

Caching

Store results of expensive reads closer to the reader.

• Where: browser, CDN, application memory, a distributed cache (Redis/Memcached), the database's own buffer.

• Strategies:

  • Cache-aside (lazy loading): read cache → on miss read DB → fill cache. Most common.
  • Write-through: write to cache and DB together; cache always fresh, writes slower.
  • Write-back: write to cache, flush to DB later; fast but risk of loss.
• Eviction: LRU (least recently used), LFU, TTL expiry.
• Problems: stale data (invalidate on writes, short TTLs), cache stampede (many misses at once: lock or serve stale while refreshing), hot keys (replicate or add local caches).

CDN

A content delivery network caches static files (images, JS, CSS, videos) and cacheable pages on servers near users. It cuts latency and offloads your origin.

Message queues and streams

Decouple producers from consumers so slow work happens asynchronously and spikes are absorbed.

• Queues (RabbitMQ, SQS, BullMQ): each message processed by one worker; retries, dead-letter queues.
• Logs/streams (Kafka): ordered, replayable events read by many consumer groups; for event-driven systems and analytics pipelines.

Use them for emails, notifications, video processing, order pipelines and syncing search indexes. Design consumers to be idempotent: messages can be delivered more than once.

Other blocks

• Object storage (S3 and similar) for files and uploads; store only the URL in the database.
• Search engine (Elasticsearch/OpenSearch) for full-text search and filters.
• API gateway: one entry point with auth, rate limiting and routing to services.
• Rate limiter: token bucket or sliding window counters in Redis.

Check yourself

• Why must app servers be stateless behind a load balancer?
• Explain cache-aside and one way it can serve stale data.
• When would you choose Kafka over a simple job queue?