Design a Price Tracking Service

#Problem Statement

Design a service that tracks prices of products across e-commerce sites and alerts users when a product's price drops below their target. Think CamelCamelCamel for Amazon or Honey.

📖

Real-world: Price tracking is a web-scraping problem with an adversary: the e-commerce sites you're scraping actively don't want to be scraped — they deploy bot detection, rate limits, CAPTCHAs, and dynamic/obfuscated HTML, and prices are increasingly rendered by JavaScript, so you need headless browsers and rotating proxies. That cat-and-mouse is the real engineering, plus change detection (poll efficiently — don't re-fetch a million products every minute; prioritize by volatility and user demand) and alerting on threshold crossings. The cautionary tale for this audience is Honey (2020+ controversy): the price/coupon tool was accused of hijacking affiliate-commission cookies (replacing the referrer's tracking code with its own at checkout) — a vivid reminder that the client-side extension form of these tools has serious supply-chain and trust implications (a browser extension with access to every page you visit is a huge attack surface). Design-wise it's scheduler + scraper pool → price-history TSDB → alert queue, but the scraping arms race and the extension's privilege are the parts worth discussing.

#Requirements

#Functional

• Track price of a product URL
• Set a price alert (notify me when price ≤ target)
• View price history (time-series chart)
• Scheduled price checks (every N hours)
• Support multiple retailers (Amazon, Walmart, Best Buy, etc.)

#Non-Functional

• 10M tracked products, 50M alerts
• Price check frequency: hourly for popular products, daily for less-popular
• Alert delivery: < 5 minutes after price drop detected
• Scraping: must handle bot detection, CAPTCHAs, rate limiting by retailers

#Core Design

#Price Scraping Architecture

Scheduler (priority queue by next_check_time)
           ↓
   Fetch Queue (Kafka, partitioned by retailer_domain)
           ↓
   Scraper Workers (per domain — respect rate limits)
           ↓
   ┌────────────────────────────────┐
   │ Parse price from HTML/JSON API │
   │ Extract: price, currency, in-stock │
   └────────────────────────────────┘
           ↓
   Price Event (product_id, price, timestamp) → Kafka
           ↓
   Price DB (TimescaleDB / Cassandra: time-series)
           ↓
   Alert Checker → Notification Service

#Scraping Challenges

Retailers actively block scrapers. Mitigation strategies:

#Alert Evaluation

On new price event (product_id, new_price):
  Query alerts WHERE product_id = ? AND target_price >= new_price AND status = 'ACTIVE'
  For each matching alert:
    if last_notified < (now - 24h):  -- don't spam if price keeps dropping
      send notification
      update last_notified

At scale (50M alerts, 10M products): alert lookup is frequent. Index (product_id, target_price). Pre-compute per-product: "what's the lowest alert target?" — only query alerts if new price ≤ that floor.

#Price History (Time-Series)

TimescaleDB (or Cassandra):
  price_history(
    product_id  UUID,
    checked_at  TIMESTAMP,
    price       DECIMAL(10,2),
    currency    CHAR(3),
    in_stock    BOOLEAN
    PRIMARY KEY (product_id, checked_at DESC)
  )

Hypertable with time partitioning. Query last 90 days fast; older data compressed/archived.

#Architecture

Scheduler → Fetch Queue (Kafka) → Scraper Workers
                                         ↓
                                  Price DB (TimescaleDB) ← Price History API
                                         ↓
                                  Alert Checker (stream processing)
                                         ↓
                                  Notification Service → Email / Push / SMS

#Security Considerations

#Interview Tips