Architect a large-scale real-time recommendation system with data pipelines

What's really being asked

This question tests whether you can design a distributed machine learning serving architecture that balances personalization, throughput, and strict latency budgets. Interviewers want to see that you understand the multi-stage funnel used by Netflix and Spotify, and that you can reason about data pipelines, embedding stores, and business guardrails rather than treating recommendation as a single model call.

The full answer

First, separate the system into candidate generation, ranking, and re-ranking stages. For candidate generation, describe two-tower neural networks as the industry standard: one tower encodes the user, the other encodes the item, and training produces embeddings that allow approximate nearest neighbor search to retrieve hundreds of candidates in sub-millisecond time across millions of items. Second, explain the ranking stage, where a lightweight model scores the retrieved candidates using real-time user features and content features, followed by re-ranking that applies business guardrails such as diversity, freshness, and exploration versus exploitation. Third, discuss data pipelines: a batch pipeline for historical training data and user profile snapshots, and a streaming pipeline to capture clicks, views, and purchases in near real time so the system adapts to changing preferences. Fourth, justify database choices with a polyglot approach: use a key-value store or wide-column database for user profiles and content metadata, a vector database or ANN index for embedding retrieval, and a caching layer for hot embeddings and precomputed recommendations to keep p99 latency under 200 milliseconds. Fifth, mention A/B testing infrastructure and horizontal scalability to handle traffic spikes while maintaining 99.9 percent uptime.

The mistakes people make

A major red flag is proposing a single monolithic model that scores every item in the catalog for each user request. At YouTube scale, where billions of videos must be considered, this is computationally impossible within a 200ms budget. Another red flag is ignoring the cold-start problem for new users or new content, or failing to balance exploration with exploitation. Finally, suggesting only one database type for all concerns shows a lack of understanding of the differing access patterns between metadata lookups, vector search, and event ingestion.

What usually comes next

Expect the interviewer to ask how you would handle a user with no history, how you would update recommendations within seconds of a major event, or how you would debug a sudden drop in click-through rate. They may also ask for a concrete latency breakdown of the funnel stages, or how you would enforce guardrails like preventing filter bubbles without sacrificing engagement.

A concrete example

Netflix evaluates 100 million user profiles against thousands of titles every second, and its recommendation system drives 80 percent of viewing hours. YouTube ranks billions of videos for over 2 billion monthly users, processes 500 hours of video uploaded every minute, and generates 70 percent of total watch time from recommendations. Amazon surfaces products from a catalog of 350 million items, and its recommendation engine drives 35 percent of revenue. These numbers illustrate why approximation, caching, and multi-stage filtering are non-negotiable in production.