Design WhatsApp (Chat System)

#Problem Statement

Design a real-time messaging system supporting 1-on-1 and group chats. Messages should be delivered reliably, with delivery/read receipts, and users should see typing indicators.

📖

Real-world: WhatsApp is the other famous "absurd scale per engineer" story — in 2014 it served 450M users with ~32 engineers, built on Erlang (whose lightweight processes and battle-tested telecom heritage are ideal for millions of persistent connections), reportedly pushing ~2M+ TCP connections per server. The design lesson is that chat is fundamentally about long-lived connections + a fan-out/delivery problem, not request/response — hence WebSockets and per-user inboxes. For a security audience the headline is end-to-end encryption: WhatsApp adopted the Signal Protocol (Double Ratchet), which means the server relays ciphertext it can't read — a profound design constraint, because features people expect (server-side search, multi-device sync, abuse scanning) all become hard when the server is blind. That tension between E2EE and trust-and-safety is a great thing to raise.

#Requirements

#Functional

• 1-on-1 messaging
• Group chats (up to 512 members)
• Message delivery/read receipts (✓ sent, ✓✓ delivered, ✓✓ read)
• Online presence indicators
• Typing indicators
• Media sharing (images, files)
• Message history

#Non-Functional

• 2B users, 100M DAU
• 50 messages/day/user → 50B messages/day → 578k messages/s
• P99 message delivery latency: < 500ms
• Messages stored forever (or configurable retention)
• End-to-end encryption (E2EE)

#Core Design

#Connection Layer — WebSocket

HTTP is request/response — server can't push without client polling. For real-time messaging, use WebSockets (persistent bidirectional TCP connection).

Client A ──WebSocket──→ Chat Server A
Client B ──WebSocket──→ Chat Server B
                    ↑
          Chat Server A → Message Queue → Chat Server B → Client B

Each user connects to one chat server. Chat servers are stateful (maintain WebSocket connections). A message routing layer (pub/sub or a message queue) connects chat servers.

#Message Flow

1. Client A sends message via WebSocket to Chat Server A
2. Chat Server A:
   a. Persist message to Message DB
   b. Publish to pub/sub channel for conversation_id
3. Chat Server B (where Client B is connected) receives from pub/sub
4. Chat Server B delivers message to Client B via WebSocket
5. Client B sends "delivered" ACK
6. Chat Server B updates message status, notifies Client A

#Offline Message Delivery

If recipient is offline:

1. Message persisted to DB
2. Push notification sent (APNs/FCM)
3. On next connection, client fetches unread messages

#Architecture

             Load Balancer (sticky sessions / consistent hash by user_id)
                    ↓
           Chat Server Cluster (WebSocket servers, stateful)
                    ↓
            ┌───────────────────────────────┐
            │                               │
     Message Queue (Kafka)         Presence Service (Redis)
            │                               │
     Message DB (Cassandra)       Group Service
            │
     Media Store (S3 + CDN)

#Database Design (Cassandra)

Cassandra is ideal: massive write throughput, time-series access pattern per conversation.

messages(
  conversation_id  UUID,
  message_id       BIGINT,     -- Snowflake timestamp-based for ordering
  sender_id        UUID,
  content          TEXT,       -- encrypted blob
  message_type     ENUM,       -- text, image, video, etc.
  created_at       TIMESTAMP,
  status           ENUM,       -- sent, delivered, read
  PRIMARY KEY (conversation_id, message_id DESC)
)

Conversation ID

• 1-on-1: min(user_a, user_b) + max(user_a, user_b) → deterministic, no coordination
• Group: generated UUID at group creation

#Presence Service

Redis:
  online:{user_id} → last_heartbeat_timestamp (TTL 30s)

• Client sends heartbeat every 10s
• If key expired → user is offline
• Typing indicators: typing:{conversation_id}:{user_id} with 5s TTL

#Group Messaging at Scale

For groups of 512 members, fan-out on write would mean 512 WebSocket pushes per message. This is manageable at small group size.

For super-groups (thousands of members) → fan-out on read: server fetches new messages on client reconnect / scroll.

#End-to-End Encryption (E2EE)

WhatsApp uses the Signal Protocol:

Key Registration:
  Client → Upload public key bundle → Key Server

Message Send (Client A → Client B):
  1. Fetch Client B's public key bundle from Key Server
  2. Perform X3DH key agreement → shared secret
  3. Encrypt message with shared secret (AES-256-GCM)
  4. Send encrypted ciphertext to Chat Server
  5. Chat Server stores + routes ciphertext
  6. Client B decrypts with their private key

Design implicationserver cannot read messages, cannot provide message search across all conversations, cannot provide server-side spam filtering of message content.

#Security Considerations

#Interview Tips