Design a system to suggest internal links for new articles

WHAT THIS TESTS: This question evaluates whether you can decompose a large-scale recommendation problem into an offline indexing stage and an online retrieval stage. Interviewers want to see that you understand embedding-based search, candidate generation versus ranking, and operational constraints like latency and index freshness. The core challenge is not just finding related pages but doing so across millions of documents with sub-second latency.

A GOOD ANSWER COVERS: First, an offline pipeline that ingests existing articles, extracts meaningful chunks or passages, and computes dense embeddings using a model like BERT or a domain-tuned sentence transformer. These embeddings are loaded into an approximate nearest neighbor index such as FAISS, HNSW, or a managed vector database. Second, an online service that takes the draft article, computes its embedding, queries the ANN index to retrieve a few hundred candidates, and then applies a lightweight reranker. The reranker should combine multiple signals in order: semantic similarity between the new article and candidate page; anchor-text suitability, meaning whether the candidate page contains a natural phrase that fits the new article's sentence structure; page authority or traffic data to prioritize high-value destinations; and editorial guardrails such as excluding outdated pages or enforcing category diversity. Third, infrastructure awareness, including periodic index rebuilds, embedding model versioning, and a fallback to keyword-based retrieval when vector search fails.

COMMON WRONG ANSWERS: Proposing to run a full-text SQL query over the entire corpus for every new article ignores latency and scale. Suggesting a single monolithic model that both embeds and scores everything in one pass usually reveals a lack of systems thinking. Ignoring the difference between candidate retrieval and ranking is a frequent red flag, as is forgetting to mention how you would prevent linking to low-quality or irrelevant pages.

LIKELY FOLLOW-UPS: How would you handle a corpus of ten million pages and keep the index fresh daily? What would you do if the suggested anchor text does not appear naturally in the target page? How would you measure success, and would you optimize for click-through rate or publisher workflow efficiency? Can you sketch the data flow from the content management system to the suggestion API?

ONE CONCRETE EXAMPLE: Imagine a news site publishing an article about electric vehicle battery recycling. The offline pipeline has already embedded every existing article. The online service generates an embedding for the new draft, retrieves the top two hundred candidate articles from the ANN index, and then reranks them. A 2022 article titled Lithium-Ion Battery Lifecycle scores highly on semantic similarity and contains the exact phrase battery recycling, so it surfaces as the top suggestion. A weaker candidate, a general page about electric cars, is demoted because it lacks specific anchor-text overlap and has lower page authority.