Design a Payment System

#Problem Statement

Design a payment processing system that handles financial transactions between payers and payees. Transactions must be reliable, auditable, and exactly-once — no double charges.

📖

Real-world: Payments run on a 700-year-old idea: double-entry bookkeeping (every transaction is two balanced entries — debit one account, credit another — so the books always reconcile and you can prove where money went). Modern systems make the ledger append-only and immutable for exactly the audit reasons a security person cares about. The defining engineering challenge is idempotency: networks retry, users double-click, and "charge $50" must happen exactly once even if the request arrives three times — solved with a client-supplied idempotency key plus a dedup table, the same pattern Stripe popularized. Multi-step flows (charge card → update ledger → notify merchant) can't use a single ACID transaction across services, so you use the Saga pattern with compensating actions to roll back. Security framing: payments are where strong consistency, immutability, idempotency, and auditability all become non-negotiable at once — "eventually consistent" loses lawsuits, and PCI-DSS scope shapes the whole architecture.

#Requirements

#Functional

• Initiate a payment (payer → payee, amount, currency)
• Payment status tracking (pending, processing, completed, failed)
• Refund capability
• Transaction history
• Support for multiple payment methods (card, bank transfer, wallet)

#Non-Functional

• Exactly-once execution — no double charges under any failure scenario
• Consistency: strong — money must balance (debit = credit)
• Durability: no lost transactions even under node failure
• Latency: < 2s for card payment end-to-end
• 1M transactions/day → ~12 TPS average; 100 TPS peak
• Financial audit trail: immutable, append-only

#Core Design

#The Fundamental Problem: Distributed Failure

Money involves at minimum two systems: your ledger and the external payment processor (Stripe, bank). What happens if:

• You debit the user but the external payment API call fails?
• The external payment succeeds but your database write fails?
• The service crashes between the two operations?

Solutionidempotency + distributed transaction patterns.

#Idempotency Keys

Every payment request must carry an idempotency key — a unique ID generated by the client.

POST /payments
{
  "idempotency_key": "client-uuid-1234",
  "amount": 9999,
  "currency": "USD",
  "payer_id": "user_a",
  "payee_id": "merchant_b"
}

Server-side:

1. Check idempotency store: has client-uuid-1234 been processed?
2. If yes: return stored result (don't process again)
3. If no: process and store result atomically

This makes retries safe: client can retry a failed request N times without double-charging.

#Double-Entry Ledger

Financial systems use double-entry bookkeeping: every transaction has a matching debit and credit. Ledger always balances.

ledger_entries(
  id            BIGSERIAL PRIMARY KEY,
  transaction_id UUID NOT NULL,
  account_id    UUID NOT NULL,
  amount        BIGINT NOT NULL,    -- positive = credit, negative = debit, in cents
  currency      CHAR(3) NOT NULL,
  entry_type    ENUM('debit','credit'),
  created_at    TIMESTAMP NOT NULL  -- immutable, append-only
)

-- Payment: $50 from Alice to Bob
INSERT INTO ledger_entries VALUES
  (txn_id, alice_account, -5000, 'USD', 'debit'),   -- Alice loses $50
  (txn_id, bob_account,  +5000, 'USD', 'credit');   -- Bob gains $50

Balance of an accountSELECT SUM(amount) FROM ledger_entries WHERE account_id = ?

This is inherently audit-proof: no rows are updated or deleted, only inserted.

#Transaction State Machine

PENDING → PROCESSING → COMPLETED
              ↓
           FAILED → (refund creates a new COMPLETED reversal transaction)

Persistence guarantees:

• Create payment record in DB (PENDING) before calling external processor
• Call external processor (Stripe/bank)
• Update record to PROCESSING (storing external transaction ID)
• On callback/webhook: update to COMPLETED or FAILED

#Saga Pattern for Distributed Transactions

When a payment spans multiple services (e.g., deduct from wallet, charge card, credit merchant), use the Saga pattern instead of distributed 2PC.

Saga: ChargeCard → UpdateWallet → CreditMerchant → NotifyUser

If CreditMerchant fails:
  Compensating transactions:
    → RevertWallet (compensate UpdateWallet)
    → RefundCard (compensate ChargeCard)

Each step is atomic in its own DB. Compensating transactions undo side effects. This achieves eventual consistency without distributed locking.

#Architecture

Client → API Gateway → Payment Service
                            |
                   ┌────────────────────┐
                   │                    │
            Idempotency Store    Payment DB (Postgres)
                (Redis)          [payments, ledger_entries]
                   |
            External Processor
                (Stripe / ACH)
                   |
            Webhook Handler    ← Stripe sends async callbacks
                   |
            Notification Service
            Reconciliation Service (daily batch: check DB vs Stripe)

#Webhook Handling

External payment processors send webhooks (HTTP callbacks) on payment completion. These must be:

@app.post("/webhooks/stripe")
def stripe_webhook(request):
    # Verify signature
    sig = request.headers["Stripe-Signature"]
    stripe.Webhook.construct_event(request.body, sig, STRIPE_WEBHOOK_SECRET)
    
    # Store event ID for idempotency
    if not idempotency_store.exists(event.id):
        idempotency_store.set(event.id, True, ttl=24*3600)
        message_queue.publish("payment_events", event)
    
    return {"status": "ok"}  # ACK immediately

#Key Design Decisions

#Security Considerations

#Interview Tips