Hybrid search and re-ranking for retrieval

WHAT THIS TESTS The interviewer wants to confirm you understand the limits of single-stage dense retrieval and can describe concrete, layered improvements with their cost trade-offs.

A GOOD ANSWER COVERS Dense bi-encoder retrieval embeds query and documents independently and matches by vector similarity, which captures semantics but blurs exact tokens like product codes, acronyms, or rare names, and can miss literal keyword matches. Hybrid search runs a sparse lexical retriever such as BM25 alongside the dense one and fuses the results, often with reciprocal rank fusion, so exact-term and semantic signals complement each other. Re-ranking adds a second stage: the first-stage retriever returns a generous top-k, then a cross-encoder reads each query-document pair together and produces a precise relevance score. Cross-encoders are far more accurate than bi-encoders because they attend across query and document jointly, but they are too slow to run over the whole corpus, so they are applied only to the shortlist.

COMMON WRONG ANSWERS Saying bigger embedding models alone solve keyword mismatch, or describing re-ranking as just another bi-encoder pass. The power of re-ranking comes precisely from joint cross-attention, which a bi-encoder lacks. Another error is ignoring latency: re-ranking a huge candidate set is prohibitively slow.

LIKELY FOLLOW-UPS Expect questions on reciprocal rank fusion weighting, on choosing k for the re-rank shortlist, on latency budgets, and on when query rewriting or HyDE helps complex queries.

ONE CONCRETE EXAMPLE A user searches for error code XJ-204. Dense retrieval returns vaguely related troubleshooting text but misses the exact page, while BM25 nails the literal code. Hybrid fusion surfaces both, then a cross-encoder re-ranker promotes the precise page to the top, combining semantic recall with lexical and re-ranked precision.