Design File Storage (Dropbox/Google Drive)

#Problem Statement

Design a cloud file storage and sync service where users can upload files, access them from any device, and share them with others. Files should sync automatically across devices.

📖

Real-world: Dropbox's core trick is content-addressed chunking with deduplication — files are split into ~4MB blocks, each hashed, and only changed blocks are uploaded (delta sync). The famous payoff: if a million users store the same popular file, it's stored once. But that same dedup created a notorious security subtlety — if dedup is global across users and the system tells you "already uploaded" for a block you didn't have, an attacker can confirm whether a specific file exists in someone else's storage (a side channel), which is why providers dedup carefully and per-account where privacy matters. Also famous: Dropbox's 2011 four-hour auth bug where any password would unlock any account, and its later move to client-side encryption debates (if the server holds the keys, it — and anyone who breaches it — can read your files). The security framing interviewers reward: who holds the encryption keys, and what does dedup leak?

#Requirements

#Functional

• Upload and download files
• Folder hierarchy
• File sync across devices (bidirectional)
• File sharing (link sharing + permission control)
• File versioning / history
• Mobile and desktop clients

#Non-Functional

• 500M users; 1B files
• Average file size: 1MB; P99: 100MB
• Files should sync within 30 seconds of change
• Storage optimized: deduplication across users (same file content = stored once)

#Core Design

#Chunked Upload

Large file uploads over a single HTTP connection fail on network interruption. Solution: chunk the file.

Client splits file into 4MB chunks
Each chunk: hash(chunk_content) → content-addressed ID
Upload: PUT /chunks/{hash} for each chunk
Commit: POST /files { chunk_hashes: [...], path: "/Documents/report.pdf" }

Benefits

• Resumable: on failure, resume from last missing chunk
• Deduplication: if chunk hash already exists in storage, skip upload (content-addressed)
• Delta sync: on file modification, only upload changed chunks (typically 1-2 chunks out of many)

#Deduplication

Two users upload the same file (e.g., Ubuntu ISO):

• Content hash of file matches a stored file → store metadata pointing to existing content
• No duplicate storage

chunks table:
  hash (PK), size, s3_key
  -- stored once even if 10,000 users have the same file

file_chunks table:
  file_id, chunk_sequence, chunk_hash
  -- mapping: file → ordered list of chunks

#Sync Protocol

Client change → Local event queue (file watcher: inotify / FSEvents)
              → Compute changed chunks
              → Upload new/modified chunks
              → POST file metadata to Sync Service
              → Sync Service publishes change event
              → Other clients for same user poll / receive WebSocket notification
              → Client downloads changed chunks only

#Architecture

Client (Desktop/Mobile)
  - File watcher (inotify/FSEvents)
  - Local chunk cache
          ↓
Upload Service → S3 (chunk storage, content-addressed by hash)
Metadata Service → PostgreSQL (files, folders, permissions, versions)
Sync Service → WebSocket / SSE (notify clients of changes)
Download Service → CDN-fronted S3 (fast downloads)
Sharing Service → PostgreSQL (share links, permissions)

#File Versioning

file_versions(
  file_id       UUID,
  version       INT,
  created_at    TIMESTAMP,
  created_by    UUID,
  chunk_hashes  TEXT[],   -- ordered array of chunk hashes
  size          BIGINT
)

Restoring a version: read chunk_hashes array, reassemble from S3. Since chunks are content-addressed, no additional storage if a restored version shares chunks with the current version.

#Key Design Decisions

#Security Considerations

#Interview Tips