Design a Chat System (WhatsApp)
Realtime 1:1 and group messaging with delivery/read receipts, presence, and offline delivery.
Overview
A chat system delivers messages between users in real time, guarantees delivery even when recipients are offline, and shows presence and receipts. The hard parts are maintaining millions of persistent connections, routing a message to the right server, and ordering.
Requirements
Functional
- 1:1 and group messaging in real time.
- Delivery + read receipts (sent / delivered / read).
- Online/last-seen presence.
- Offline users receive messages when they reconnect; message history persists.
Non-functional
- Low end-to-end latency (< 200 ms when both online).
- At-least-once delivery with client-side dedup; consistent ordering within a conversation.
- Scale to hundreds of millions of concurrent long-lived connections.
Back-of-the-envelope
The numbers that justify the architecture.
| DAU | 500M | |
| Messages / day | 50B | ~600K msgs/sec avg |
| Concurrent connections | ~100M | sizing the WS fleet |
| Avg message size | ~100 bytes |
Reference architecture
Stateful WebSocket gateways hold connections; a session registry maps user→gateway; messages route through a queue and persist to a wide-column store.
Deep dives
Connections and routing
Clients hold a persistent WebSocket to a stateful gateway. To deliver a message from A to B, the chat service looks up which gateway currently holds B's connection in a session registry (Redis) and forwards it there. If B is on a different gateway, route via a message queue / internal RPC. This user→gateway indirection is what makes horizontal scaling of a stateful connection tier possible.
Offline delivery and ordering
Every message is persisted first, then delivered. If the recipient is offline, it sits in their per-user inbox (or a mailbox partition in Cassandra) and is pushed via APNs/FCM; on reconnect the client pulls everything after its last-seen message id. Order messages within a conversation with a monotonic sequence id assigned server-side (per-conversation), not client clocks — clocks lie. Clients dedup on message id to tolerate at-least-once redelivery.
Group messages
For small groups, fan out to each member's mailbox on send. For large groups, this is the fan-out problem again — cap synchronous fan-out and let a worker distribute, or keep a single group log that members read. Receipts multiply: 'delivered' from N members means N acknowledgements; often only aggregate counts are surfaced.
Presence
Presence is a heartbeat: clients ping periodically; the presence service marks online and expires to offline on missed heartbeats (TTL). Don't broadcast every presence change to everyone — push updates only to a user's active conversation partners to avoid an O(friends) fan-out storm.
Key trade-offs
For each decision: the two options, and when to pick which.
Bottlenecks & follow-ups
Where it breaks under load — and what an interviewer will probe.
- ▲Connection tier memory/CPU → many gateways, consistent-hash users to them.
- ▲Session registry hotspots → shard by user id; keep it in-memory.
- ▲Large-group fan-out → async workers, bounded synchronous fan-out.
What a strong answer sounds like
- ✓Draw the user→gateway session registry — it's the crux of scaling stateful connections.
- ✓Say 'persist then deliver' — durability before realtime is the correct ordering.
- ✓Handle the offline case explicitly; it's what separates a real design from a toy.