← All cheatsheets

System Design

Load Balancing: Layers, Algorithms & Failover

A load balancer spreads traffic across healthy backends. Layer 4 routes on IP/port — fast and protocol-agnostic; Layer 7 understands HTTP and can route on content, terminate TLS, and retry, at higher cost. Pick the algorithm for your traffic shape, and make the balancer itself redundant — a lone LB is just a relocated single point of failure.

Updated September 12, 2026 · 4 min read

Layer 4 vs Layer 7

Layer 4 (transport)	Routes on IP + port; forwards connections without reading the payload.Fast, cheap, protocol-agnostic (TCP/UDP). No content routing.
Layer 7 (application)	Understands HTTP: routes on path/host/header, terminates TLS, retries, rewrites.Richer, but more CPU and a decryption point.
Rule of thumb	L7 for HTTP APIs and content routing; L4 for raw throughput or non-HTTP protocols.

Balancing algorithms

Round robin	Next backend in rotation.Simple; ignores load and request cost.
Weighted round robin	Rotation biased by capacity.For heterogeneous backends.
Least connections	Fewest active connections wins.Better when request durations vary.
Least response time	Fewest connections + lowest latency.Adapts to slow backends.
Hashing / consistent hashing	Map a key (client IP, session, cache key) to a backend.Session affinity + cache locality; consistent hashing minimizes reshuffle when backends change.

Health, failover & the LB itself

Health checks	Active (probe an endpoint) or passive (watch real traffic for errors).Pull unhealthy backends out; re-add after they pass.
Connection draining	Let in-flight requests finish before removing a backend.Graceful deploys and scale-in.
Sticky sessions	Pin a client to one backend (cookie or hash).Needed for server-side session state — prefer stateless backends instead.
TLS termination	Decrypt at the L7 LB; re-encrypt to backends if needed.Centralizes certs; the LB sees plaintext.
The LB is a SPOF	Run redundant LBs (active/active or active/passive) behind a VIP, anycast, or DNS.One load balancer is one point of failure.

In the interview

Name the layer and why — “L7 here, because I need path-based routing and TLS termination” beats “add a load balancer.”
Pick the algorithm from the traffic shape: least-connections when request cost varies, consistent hashing when you need cache affinity.
Always make the balancer itself redundant — a lone LB just moves your single point of failure, it doesn’t remove it.