System Design Foundations

#The 30-Minute Interview Framework

Before you know any distributed systems concept, know this framework cold. It structures how you present any design.

0–3 min   → Clarify requirements
              Ask: Who uses this? What are the core actions? What scale?
              Produce: Functional requirements (features) + Non-functional (scale, latency, availability)

3–5 min   → Capacity estimation
              Compute: QPS (reads + writes), storage, bandwidth
              Rule of thumb: 1M req/day ≈ 12 QPS; 1B req/day ≈ 11,500 QPS

5–10 min  → High-level design
              Draw the main components: clients, API gateway, services, DB, cache
              Describe the happy path end-to-end

10–20 min → Deep dive on the hard problems
              Pick 2–3 interesting challenges and go deep
              Example: "The hard part is seat reservation under concurrency — let me explain"

20–25 min → Security, failure modes, and trade-offs
              Always mention: TLS, AuthN/AuthZ, rate limiting, audit logging
              Failure: what happens if the DB goes down? If the cache is cold?

25–30 min → Wrap up and questions

The single most important habitstate your assumptions out loud. "I'm assuming 100M daily active users, 90/10 read/write ratio — does that match what you're thinking?" This shows you reason systematically, not just name-drop technologies.

#Numbers Every Engineer Should Know

These numbers let you do back-of-envelope math in your head. You don't need them exact — within an order of magnitude is fine.

#Latency Reference Points

What this means in practiceif a user action requires reading from disk (HDD) instead of cache (RAM), it's 100,000× slower. This is why caching is not optional — it's the difference between a 100 µs response and a 10 ms one.

#System Throughput

#Size Reference Points

#Quick Estimation Math

1 million requests/day  ÷ 86,400 sec/day ≈ 12 QPS
1 billion requests/day  ÷ 86,400 sec/day ≈ 11,500 QPS

Twitter example:
  500M tweets/day at 280 bytes each
  = 500M × 280 bytes = 140 GB of raw tweet data per day
  = 51 TB/year — needs distributed storage + archival strategy

YouTube example:
  500 hours of video uploaded per minute
  At 4 GB/hour = 2,000 GB = 2 TB uploaded per minute
  Need parallel upload pipelines and massive object storage

#CAP Theorem — The Fundamental Trade-off

The concept in one sentencea distributed system that experiences a network partition (some nodes can't talk to each other) must choose between staying available (accepting requests) or staying consistent (refusing requests until the partition heals).

#Why Partition Tolerance Is Non-Negotiable

Imagine you have two database servers, Server A in New York and Server B in London, and the network cable between them is cut. Requests are still arriving. What do you do?

• You cannot pretend the partition isn't happening — if you do, Server A and Server B will accept conflicting writes and diverge.
• So you must choose: do you keep serving requests (available but potentially inconsistent) or stop serving until the link is restored (consistent but unavailable)?

This is the CAP choice. Every distributed system faces this.

CP — Choose Consistency over Availability
  When the network link breaks: Server B refuses all writes.
  Users get errors. Bank balances stay correct.
  Use this for: banking transactions, inventory counts, seat reservations,
                anything where two conflicting answers is worse than no answer.
  Examples: HBase, Zookeeper, etcd

AP — Choose Availability over Consistency
  When the link breaks: both servers keep accepting writes.
  Users get responses. Some users see stale data for a while.
  Use this for: social media feeds, DNS, caches, shopping recommendations,
                anything where "slightly wrong" is better than "unavailable".
  Examples: DynamoDB, Cassandra, CouchDB, DNS

The rule of thumb for interviewsask "what's worse — a stale answer or no answer?"

• If stale answer is worse (money, inventory, reservations) → CP
• If no answer is worse (social content, search, recommendations) → AP

#Consistency Models

Not all "consistent" is the same. These are the levels from weakest (most scalable) to strongest (most correct).

#The Spectrum

WEAKEST  ←────────────────────────────────────────────────────────────→  STRONGEST
Eventual    Monotonic Read    Read-Your-Writes    Session    Bounded Staleness    Linearizable

Eventual Consistency — the weakest. Eventually all replicas will agree, but you don't know when.

Real-world example: You like a Facebook post. Your friend in another city might not see the like count increase for a second or two. That's fine — nobody cares if the count is off by 1 for 200ms.

Monotonic Read — you'll never see an older version than the one you already read.

Real-world example: You refresh your Twitter timeline and see 200 tweets. If you refresh again, you won't suddenly see 190 tweets (as if the newer 10 were erased). You might not see new tweets yet, but you won't go backward.

Read-Your-Writes — you always see the effects of your own writes, even if others don't yet.

Real-world example: You update your profile photo on LinkedIn. You immediately see your new photo. Other users might still see the old one for a few seconds — but you always see your own change instantly.

Session Consistency — within a single user session, reads are consistent.

Real-world example: Your shopping cart. Within one browser session, the items you added are always visible. If you open a new browser (new session), there might be a brief sync delay.

Bounded Staleness — data is at most N seconds (or N versions) behind.

Real-world example: A news site CDN caches articles for 30 seconds. You might see an article that's up to 30 seconds old, but never older. Used in Azure Cosmos DB's strong-bounded mode.

Linearizability (Strong Consistency) — every read reflects the most recent write. Behaves as if there's only one copy of the data.

Real-world example: A bank account balance. Every ATM withdrawal must see the exact current balance — not a cached version from 200ms ago. If two ATMs process withdrawals simultaneously on a $100 account, exactly one must see $100 and one must see $0 (or fail). No double-spend. Used by: Google Spanner, single-node PostgreSQL, etcd (for cluster coordination).

#Database Selection Guide

The most common interview mistake: reaching for PostgreSQL for everything. Different storage problems need different storage engines.

#The Decision Tree

Do you need transactions across multiple tables?
  Yes → Relational (PostgreSQL, MySQL)

Is the data write-heavy with simple access patterns (no complex queries)?
  Yes → Wide-column (Cassandra, HBase) or Time-series (InfluxDB, TimescaleDB)

Do you need full-text search?
  Yes → Search engine (Elasticsearch) as a secondary store, not primary DB

Is this a cache/session store?
  Yes → Key-value in-memory (Redis, Memcached)

Are you storing files, images, video?
  Yes → Object storage (S3, GCS)

Do you have a deeply connected graph (social network, fraud detection)?
  Yes → Graph DB (Neo4j, Amazon Neptune)

Otherwise → start with PostgreSQL and shard/switch later

#Reference Table

#Caching

Why caching exists in one sentenceRAM is 1000× faster than SSD and 100,000× faster than a HDD. If you can answer a request from RAM instead of disk, do it.

#The Cache Hit / Miss Cycle

Request arrives for user_id=123's profile:

CACHE HIT (fast path):
  App → check Redis → key exists → return data in ~1ms
  No database involved.

CACHE MISS (slow path):
  App → check Redis → key not found
  → query PostgreSQL → return data in ~20ms
  → write result to Redis with TTL=300s
  → return data to user
  Next request for same user → cache hit

#Caching Patterns

Cache-aside (most common) — the application manages the cache manually.

Read: App asks cache → miss → App asks DB → App writes to cache → returns to user
Write: App writes to DB → (optionally) deletes or updates cache entry

Use this when: read-heavy workloads, you're retrofitting caching onto an existing system.

Risk: cache stampede — if 10,000 users all request the same key at the same moment it expires, all 10,000 go to the database simultaneously. Mitigation: mutex lock (first miss takes a lock, others wait), or probabilistic early expiration (randomly refresh before TTL expires to spread the load).

Write-through — every write goes to cache AND database, synchronously.

Write: App writes to cache → cache writes to DB → returns success
Read:  Cache always has fresh data → simple read

Use this when: writes and reads are balanced, freshness is critical. Trade-off: higher write latency (two writes instead of one).

Write-behind (write-back) — writes go to cache immediately, database is updated asynchronously.

Write: App writes to cache → returns success immediately
Background: cache flushes to DB asynchronously every N seconds

Use this when: you need very low write latency and can tolerate small risk of data loss. Risk: if the cache server crashes before flushing, those writes are lost.

#Cache Eviction Policies

When the cache is full, which items get removed?

Redis defaultallkeys-lru (when memory is full, evict the least recently used key from all keys).

#Cache Invalidation — The Hard Problem

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

How do you keep the cache consistent with the database?

#Scaling Patterns

#Vertical vs Horizontal Scaling

The interview answerstart with vertical scaling (it's simpler, no code changes needed). Switch to horizontal when you hit the hardware ceiling or need geographic redundancy.

#Load Balancing

A load balancer sits in front of your servers and distributes incoming requests. Without it, one server gets everything and the others are idle.

                   ┌─→ Server A (handles 33% of traffic)
Users → LB ────────┼─→ Server B (handles 33% of traffic)
                   └─→ Server C (handles 33% of traffic)

Algorithms

L4 vs L7 load balancers

#Database Scaling

Read replicasthe simplest horizontal DB scaling. All writes go to the primary. Reads spread across multiple replicas.

Writes → Primary DB
          ↓ replication (async)
Reads  → Replica 1, Replica 2, Replica 3

Trade-off: replica lag. A replica might be 50–200ms behind the primary. Users who just wrote data might read stale data from a replica. Fix: route writes and reads-after-write to the primary; route all other reads to replicas.

Shardingsplit the database into multiple independent databases (shards), each owning a portion of the data.

user_id 1–1,000,000    → Shard 1 (its own PostgreSQL instance)
user_id 1,000,001–2M   → Shard 2
user_id 2,000,001–3M   → Shard 3

Why it's complex: JOINs across shards are expensive or impossible. Rebalancing when a shard gets too large is painful. Pick the shard key carefully — if everyone queries by user_id, shard by user_id. Don't shard until you absolutely need to.

CQRS (Command Query Responsibility Segregation)instead of one database for both reads and writes, have two separate models. Writes go to a normalized relational DB. Reads go to a denormalized "read model" optimized for queries (could be a different database type — Elasticsearch, Redis, a pre-aggregated table).

Real-world example: Twitter's tweet storage (Cassandra for writes) vs tweet search (Elasticsearch for full-text search). Two different systems, kept in sync via an event stream.

#Message Queues

#Why Queues Exist

Imagine a user uploads a photo to Instagram. Three things need to happen: resize the photo to multiple resolutions, run it through content moderation, and notify followers. You could do all three synchronously — but then the upload API call takes 5 seconds. Or you can:

1. Accept the upload, save the original photo.
2. Return success to the user immediately.
3. Push a "photo uploaded" event to a queue.
4. Three separate worker services consume the event and do their jobs independently.

The user sees instant success. The heavy work happens in the background, decoupled.

What queues give you:
  Decoupling:       Producer doesn't know or care about consumers
  Spike absorption: Consumer works at its own pace; queue buffers the backlog
  Retry:            Failed jobs can be retried without bothering the producer
  Fan-out:          Multiple consumers can process the same event (photo resize + moderation)

#Kafka vs RabbitMQ vs SQS

When to pick Kafkaevent streaming, event sourcing, systems that need to replay events, fan-out to many independent consumers, audit logs.

When to pick RabbitMQwork queues where exactly one worker processes each job, complex routing rules, need acknowledgment guarantees.

When to pick SQSyou're on AWS and want something managed with zero ops overhead.

#Delivery Guarantees

The practical answerdesign for at-least-once delivery and make your consumers idempotent. Use a database deduplication key (message_id → processed = true) to detect and skip duplicates.

#Consistent Hashing

#The Problem It Solves

Imagine you have a cache cluster with 3 nodes (A, B, C). You decide which node holds a key with node = hash(key) % 3. This works until you add a 4th node.

With 3 nodes: hash("user:123") % 3 = 1 → Node B With 4 nodes: hash("user:123") % 4 = 3 → Node D

Almost every key now maps to a different node. Your entire cache is invalidated overnight. The database gets a sudden surge of cache misses.

Consistent hashing solves this: when you add or remove a node, only the keys that were owned by the added/removed node need to move. All other keys stay put.

#How It Works

Imagine a circle (ring) representing all possible hash values from 0 to 360 degrees. You place nodes at positions around the ring:

Ring (0° → 360°):

            Node A (at 90°)
         /
  0° ──────────── 90° ── Node A
  │                     │
  │         Ring        │
  │                     │
  Node C (270°) ─── 180° ── Node B

To find which node holds a key: hash the key → get an angle → move clockwise → first node you hit owns the key.

"user:123" hashes to 150° → move clockwise → hits Node B at 180° → Node B owns it
"user:456" hashes to 220° → move clockwise → hits Node C at 270° → Node C owns it

Adding a node (Node D at 210°):

Before: keys between 180° and 270° were owned by Node C
After:  keys between 180° and 210° → Node D (only these move)
        keys between 210° and 270° → still Node C (unchanged)

Only ~1/N of keys move when you add/remove a node (N = number of nodes). All other keys stay on the same node.

Virtual nodesreal implementations give each physical node multiple positions on the ring (e.g., Node A appears at 90°, 200°, 340°). This ensures even key distribution even with a small number of nodes, and allows fine-grained load balancing.

Used in: Redis Cluster, Apache Cassandra, Memcached (ketama), DynamoDB.

#API Design Principles

#Idempotency

An operation is idempotent if calling it multiple times has the same effect as calling it once.

Idempotent:     GET /users/123    (reading twice = same result)
                PUT /users/123    (setting the same value twice = same state)
                DELETE /users/123 (deleting twice = still deleted)

NOT idempotent: POST /orders      (submitting twice = two orders!)

Why it matters: networks are unreliable. If a payment request times out, should you retry? Only if the operation is idempotent. To make POST /payments idempotent: require an idempotency key header (Idempotency-Key: uuid). Store a mapping of {idempotency_key → result}. If the same key arrives again, return the stored result instead of processing again.

#Pagination

Never return all records at once. For large datasets:

Offset-based (simple, but slow for large offsets):
  GET /posts?page=1&limit=20
  GET /posts?page=1000&limit=20   ← DB must scan 20,000 rows to find the 1,000th page

Cursor-based (recommended for large datasets):
  GET /posts?limit=20
  Response: { "posts": [...], "cursor": "eyJpZCI6MTIzNH0=" }
  Next page: GET /posts?limit=20&cursor=eyJpZCI6MTIzNH0=
  ← DB directly jumps to the row after the cursor; no scanning

#Rate Limiting

Always design the API layer with rate limits. Two main algorithms:

Token bucketeach client has a bucket of N tokens. Each request consumes 1 token. Tokens refill at a constant rate. This allows bursting up to the bucket size.

Bucket: capacity=100, refill=10/second
User sends 100 requests at once → all allowed (uses all tokens)
11th request → rejected (bucket empty)
After 1 second → 10 tokens refilled → 10 more requests allowed

Sliding window countertrack request count in the last N seconds using two counters (current window + previous window weighted by overlap). More accurate than a fixed window, very efficient (2 integers per client).

Communicate limits to clients: Retry-After: 30, X-RateLimit-Limit: 100, X-RateLimit-Remaining: 0.

#Versioning

Always version your APIs:

URL versioning (simplest):    /v1/users, /v2/users
Header versioning:            Accept: application/vnd.api+json; version=2

Never break an existing API version. Run old and new versions in parallel during migration.

#Distributed Systems Failure Modes

These are the failure patterns you must know and explain mitigations for.

#Single Point of Failure (SPOF)

Any component that, if it fails, takes the whole system down.

Example: Your application has one Redis instance for sessions. Redis goes down → all users are logged out.

MitigationRedis Sentinel (automatic failover) or Redis Cluster (sharded + replicated). Apply the same thinking to databases, message brokers, and load balancers.

#Cascading Failure

Service A calls Service B. Service B is slow. Service A's threads pile up waiting for B. Service A runs out of threads. Now Service C, which calls Service A, is also stuck. The slowdown cascades.

Real-world example: Amazon Prime Day 2018 — a dependency bottleneck caused a cascade that brought down multiple services.

MitigationCircuit breaker pattern. After N consecutive failures or when error rate exceeds a threshold, the circuit "opens" — subsequent calls immediately fail fast (no waiting). After a timeout, the circuit tries again ("half-open"). If success, it closes. Libraries: Hystrix (Java), Polly (.NET), resilience4py.

#Cache Stampede (Thundering Herd)

A popular cache key expires. 10,000 simultaneous requests all miss and all hit the database at once.

Mitigationmutex lock on cache miss (first miss locks, populates; others wait), or probabilistic early expiration (refresh slightly before TTL expires to spread the load over time).

#Split-Brain

Two database nodes each think they are the primary. Both accept writes. Data diverges.

Example: network partition causes two PostgreSQL nodes to both believe they're the primary. After the partition heals, you have conflicting writes on both.

Mitigationquorum-based consensus (Raft, Paxos). A node can only become primary if it has acknowledgment from a majority (quorum) of nodes — impossible for two nodes to both achieve quorum simultaneously.

#Hot Partition

In a sharded database or Kafka, all traffic concentrates on one shard.

Example: you shard Kafka by user_id. Justin Bieber has 100M followers and posts constantly. His user_id partition gets 100× the traffic of any other.

Mitigationadd a random suffix to the partition key for hot users (user_id + random(0,9) → 10× spread). For Kafka, use more partitions for high-volume topics.

#Security Fundamentals for System Design

Weave these naturally into your design. Don't wait to be asked.

#Authentication vs Authorization

JWT best practicesshort-lived access tokens (15 minutes) + long-lived refresh tokens. Store refresh tokens in HttpOnly cookies (not localStorage — localStorage is XSS-accessible). Access tokens in memory only.

#The Standard Security Stack for Any System Design

Client → CDN/WAF (DDoS protection, OWASP rules)
       → API Gateway (rate limiting, auth token validation)
       → Service (AuthZ check, input validation)
       → Database (parameterized queries, column-level encryption for PII)
       → Audit log (every mutation, immutable, off to SIEM)

Mention this stack in every design, even if briefly. It signals security-aware thinking.

#Data Classification

Always ask: "what data is sensitive here?" Then apply the right controls:

Public data (posts, public profiles): no special encryption; CDN cacheable
Internal data (user emails, phone numbers): encrypt at rest (column-level AES-256)
Sensitive data (passwords, SSNs, payment cards): bcrypt/Argon2 for passwords;
    tokenize card numbers (store token, send real number only to payment processor)
Audit data: immutable, separate storage, append-only

#Audit Logging

Every system design interview should mention audit logging for any financial, healthcare, or user-data system:

What to log: actor (user/service), action (read/write/delete), resource (table + ID),
             timestamp, source IP, outcome (success/failure)
Where to store: separate append-only log database; ship to SIEM (Splunk, Chronicle)
Why separate: so a compromised application cannot delete or modify audit records

#Input Validation

At the API boundary:

• Type check, length limits, allowed character sets
• SQL injection: always use parameterized queries (? placeholders, never string concatenation)
• SSRF: if your API accepts user-supplied URLs, block requests to 169.254.169.254 (cloud metadata), 10.x.x.x, 172.16.x.x, 192.168.x.x (internal networks)
• XXE: disable XML external entity processing when parsing XML

#The Big Trade-offs

Memorize these. Every system design decision is one of them.