What is a feature store and how does it prevent training-serving skew?

WHAT THIS TESTS: The interviewer wants to know if you understand that a feature store is not just storage but a system for managing the lifecycle of machine learning features with strict consistency guarantees between training and serving. They care whether you can articulate the training-serving skew problem, explain how shared transformation logic and dual storage modes solve it, and discuss reasonable alternatives when a full feature store is overkill.

A GOOD ANSWER COVERS: First, define the feature store as a centralized repository for curated, reusable features derived from raw data. Second, explain the two storage layers: an offline store for batch historical data used during model training, and an online store optimized for low-latency lookups during real-time inference. Third, emphasize that the same transformation code runs on both paths, which prevents subtle bugs where training features are computed differently than serving features. Fourth, mention point-in-time correctness, meaning the offline store lets you retrieve feature values as they existed at specific historical timestamps to avoid data leakage. Fifth, discuss alternatives such as ad-hoc ETL pipelines maintained by individual teams, embedding feature engineering logic directly inside the model service, or relying on a plain data warehouse without dedicated serving infrastructure.

COMMON WRONG ANSWERS: A major red flag is describing a feature store as simply a database or a key-value cache for features without mentioning transformation consistency or the offline-online split. Another weak pattern is conflating a feature store with a data catalog or data warehouse; while related, a feature store specifically manages feature transformations and serving latency requirements. Candidates also stumble by ignoring point-in-time correctness, which is essential for preventing label leakage when training on historical data. Finally, suggesting that a feature store is always necessary shows a lack of architectural judgment, since small teams or simple batch models often do fine without one.

LIKELY FOLLOW-UPS: The interviewer may ask how you would handle feature backfilling when adding a new feature to the store, or how you would ensure low latency if the online store becomes a bottleneck. They might probe the trade-offs between pre-computed features in the online store versus on-demand transformations at serving time. Another common follow-up is how feature stores interact with streaming pipelines, or how you would implement feature versioning and lineage tracking to reproduce an old model exactly.

ONE CONCRETE EXAMPLE: Imagine an e-commerce recommendation model that needs a user's average purchase amount over the last thirty days. In a feature store, a batch pipeline computes this metric nightly and writes it to the offline store for training, while a real-time pipeline updates the online store as each purchase occurs. The same aggregation logic runs in both pipelines. Without a feature store, the training team might compute the average from a data warehouse using SQL with one timezone, while the serving team implements it in Python with another, causing the model to see inconsistent values in production and degrading prediction quality.