Sub-20ms online feature serving

WHAT THIS TESTS Whether you can design an online feature serving layer that meets a strict sub-twenty-millisecond p99, not just average latency.

A GOOD ANSWER COVERS The core principle is that the request path must be a fast key-value lookup, not a computation. Use an in-memory or near-memory store such as Redis, or a managed low-latency store like DynamoDB, as the online feature store, keyed by entity id, for example card or account. Precompute expensive aggregate features, like transactions in the last hour, in a streaming job (Flink, Kafka Streams, or Spark Structured Streaming) and write the current values into the online store so serving is a simple read. At inference, do one batched multi-get for all needed feature keys; compute only trivial request-time features such as amount or time-of-day in process. Co-locate the store with the inference service and use connection pooling to cut network hops.

CONSISTENCY Maintain a single feature definition that materializes both the offline training table and the online store, preventing training-serving skew. A feature store framework like Feast or Tecton enforces this.

LATENCY DISCIPLINE Measure p99, not mean; cap the number of keys fetched, avoid fan-out, set tight timeouts with sensible defaults, and warm caches.

COMMON WRONG ANSWERS Querying a data warehouse or running aggregations synchronously per request, which blows the budget, ignoring training-serving skew, or optimizing average latency while p99 spikes.

ONE CONCRETE EXAMPLE A streaming job updates per-card rolling counts in Redis on every transaction. A scoring request issues one Redis multi-get for the card's features, adds the current amount in memory, and returns a fraud score, completing the lookup in single-digit milliseconds well inside the p99 target.