How would you modify retrieval architecture for hybrid text and SQL RAG?
It tests unified retrieval across unstructured text and structured SQL. Outline a query planner that routes to vector search or text-to-SQL, joins the results, and synthesizes a final answer. Never suggest embedding the whole database as text chunks.
What's really being asked
This question probes whether you understand that unstructured text retrieval and structured data querying are fundamentally different operations with different failure modes, latency profiles, and correctness requirements. The interviewer wants to see if you can design a routing and synthesis layer rather than treating all data as embeddings. Specifically, they are looking for awareness that SQL engines handle aggregations, joins, and precise numeric filters far better than semantic search ever could.
The full answer
First, intent classification or query planning to decide whether the question needs text, SQL, or both. Second, preserving native SQL execution for structured data instead of abandoning the relational engine, because vector search cannot reliably sum columns or filter by exact dates. Third, running retrieval paths in parallel or sequence depending on dependencies, for example using the SQL result to filter which documents to retrieve, or vice versa. Fourth, a synthesis stage where both structured results and unstructured snippets are fed into the final LLM prompt with clear citations and source attribution. Fifth, handling schema exposure by providing the LLM with relevant table schemas, descriptions, and few-shot examples to improve text-to-sql generation accuracy.
The mistakes people make
The biggest red flag is suggesting you dump SQL tables into documents, embed them, and rely solely on vector search. This loses numeric precision, aggregation power, and join semantics entirely. Another weak pattern is proposing a single monolithic prompt that asks the LLM to write SQL and summarize documents simultaneously without an explicit intermediate result stage, which dramatically increases hallucination risk and makes debugging impossible. Ignoring schema context and expecting the LLM to guess table names or column types is also a serious failure mode that leads to bad queries.
What usually comes next
How do you handle cases where the SQL query returns no rows but the text corpus has relevant context, or the reverse? How do you prevent SQL injection or limit the LLM to read-only queries against a read replica? What if the user question is ambiguous and could be answered by either source alone? How do you rank or reconcile conflicting information between a document and a database record? How do you manage latency and cost when both retrievers are slow or when the SQL query requires scanning large tables?
A concrete example
Imagine a financial RAG system where a user asks how much revenue the company had last quarter and what the CEO said about it. The query planner detects both a numeric aggregation and a qualitative summary. The SQL path executes SELECT revenue FROM quarterly_reports WHERE quarter equals Q2_2024 against the finance database. The vector path retrieves the top three chunks from earnings call transcripts mentioning revenue outlook. The synthesis prompt receives the exact SQL result and the transcript snippets, then generates an answer that pairs the 450 million dollar figure with the CEOs commentary on supply chain headwinds, citing both the database row and the document source.
Interview question
Which approach best handles a user question requiring both a precise database aggregation and a qualitative summary from documents in a hybrid RAG system?
- a.Retrieve text chunks first and prompt the LLM to infer the numeric aggregation from the unstructured text instead of using SQL
- b.Use one LLM prompt that simultaneously writes the SQL query and retrieves relevant text chunks without intermediate result stages
- c.Use a query planner to route the aggregation to native SQL execution and the summary request to vector search, then synthesize both outputsCorrect
- d.Embed all database tables as text chunks and perform a single vector search across both tables and documents
Why? this is the answer
Native SQL preserves exact numeric aggregations and filter semantics that vector search or text inference cannot reliably replicate, while synthesis combines both sources accurately. Option D is the most tempting distractor because it mirrors standard text-only RAG, but flattening tables into chunks destroys relational operations like joins and SUMs.
Just read this? Test yourself on what you have been reading.
Read the original → llamaindex.ai
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on llms — each one lists the topics its interview covers.
See open roles