Design a sub-50ms real-time bidding feature pipeline

What's really being asked

The interviewer wants to see if you understand the tension between high-throughput batch historical data and low-latency streaming data in a mission-critical serving path. They are evaluating your ability to partition a problem into hot, warm, and cold paths, select appropriate storage systems by latency profile, and reason about consistency and freshness tradeoffs under a hard SLA. This is fundamentally about feature platform architecture, not just ETL.

The full answer

First, a dual-ingestion architecture. The batch path processes terabytes of historical click and conversion logs nightly or hourly, writing pre-aggregated user profiles to a low-latency key-value store such as Redis, Aerospike, or a cloud-native KV service with p99 read latency under 5 milliseconds. Second, a streaming path that ingests live impression and click events from Kafka or Pub/Sub, computes session-level aggregations in a stateful stream processor like Flink or Spark Streaming, and writes results to an in-memory cache with TTL eviction so that real-time features decay naturally. Third, a serving layer that performs parallel lookups to both stores and merges feature vectors in under 10 milliseconds, leaving headroom for network hops and model inference. Fourth, operational considerations including backfill strategies when the streaming job replays, feature versioning to prevent training-serving skew, and circuit breakers so that a cold store timeout does not blow the 50 millisecond budget.

The mistakes people make

A single-database design such as dumping everything into BigQuery, Snowflake, or Postgres and querying it per bid request. These systems have query latencies measured in seconds or hundreds of milliseconds, which instantly fails the SLA. Another red flag is ignoring the merge step entirely and assuming one storage system can handle both batch and streaming optimally. Candidates also err by proposing heavy serialization formats like JSON or XML at serving time instead of flatbuffers or protobuf, adding unnecessary parse overhead. Finally, neglecting to mention TTLs or windowing for streaming features leads to unbounded memory growth and stale data.

What usually comes next

How do you handle late-arriving events in the streaming path without corrupting the session state? What is your strategy for backfilling a new feature into the key-value store without impacting serving latency? How do you ensure exactly-once semantics for billing-related features versus at-least-once for coarse aggregations? What happens when the in-memory cache evicts a hot key during a flash sale or traffic spike? How do you monitor and alert on end-to-end feature freshness?

A concrete example

Imagine a user who has purchased running shoes in the past. The batch pipeline has precomputed a historical category affinity vector and stored it under the user ID in Aerospike with a 24-hour TTL refresh. During the current browsing session, the user clicks three ads for trail running gear within two minutes. The streaming Flink job increments a real-time trail-running intent score and writes it to Redis with a 5-minute TTL. When the ad exchange receives a bid request, the serving microservice issues parallel asynchronous GETs to Aerospike and Redis, merges the vectors into a single 200-dimensional feature payload using protobuf, and passes it to the model in 12 milliseconds total.