RAG
28 bites tagged RAG — interview questions with model answers, and 60-second explainers.
Why RAG persists despite million-token context windows
Cost and latency scale with context, attention degrades in the middle, and RAG adds freshness, access control, and citations. RAG versus long-context tradeoffs. Assuming a huge window equals reliable use of all of it.
Practical explainability for an LLM loan summary
Use attribution-by-design with grounded citations, structured rationales, and a deterministic rules layer instead of slow per-token SHAP. Pragmatic LLM explainability under constraints.
Chunking and embedding a RAG corpus
Choose chunk size and overlap balancing context vs precision, prefer semantic boundaries, then pick an embedding model matching domain and dimension, and store with metadata. practical RAG ingestion judgment.
When to choose RAG over fine-tuning
RAG for fresh, factual, citable knowledge that changes often, fine-tuning for behavior, style, or format the model must internalize. matching technique to the kind of adaptation needed.
Fine-tuning vs RAG for daily-updated docs
Choose RAG because docs change daily, embed and index chunks in a vector store, retrieve top matches and inject into the prompt. matching the right technique to freshness needs.
Detecting RAG hallucinations with a confidence score
Decompose the answer into claims, verify each against retrieved context with NLI or an LLM judge, aggregate into a faithfulness confidence score, and flag unsupported claims. groundedness verification design.
How to evaluate a RAG system end to end
Measure retrieval (recall, precision, MRR, NDCG) and generation (faithfulness, answer relevance) separately, plus end-to-end correctness. ability to decompose RAG evaluation. judging only final answer quality with one number.
Evaluating a RAG system end to end
Measure retrieval with context recall or precision, and generation with faithfulness and answer relevance, attributing failures to the right stage. ability to separate retrieval and generation quality.
Hallucination Detection in LLMs
Hallucination detection is the set of techniques for flagging when a language model states something fluent but false or unsupported, using signals like self-consistency, model uncertainty, and grounding against retrieved evidence to catch fabrications before…
How would you architect a multi-turn conversational RAG system?
This tests memory and query reformulation design beyond single-turn RAG. A strong answer covers 5-10 turn windows, LLM-based rewriting with coreference resolution, hybrid fallbacks, and summarized memory.
Identify RAG latency bottlenecks and propose optimizations
This tests systems thinking across the RAG pipeline. A strong answer names four bottlenecks—embedding, search, chunking, and generation—and pairs each with caching, index tuning, and distillation. Red flag: GPU scaling without indexing fixes.
How would you modify retrieval architecture for hybrid text and SQL RAG?
It tests unified retrieval across unstructured text and structured SQL. Outline a query planner that routes to vector search or text-to-SQL, joins the results, and synthesizes a final answer. Never suggest embedding the whole database as text chunks.
Why does your RAG ignore or contradict retrieved context?
Tests separation of retrieval failures from generation grounding in RAG. Strong answers trace symptoms to root causes like bad chunks, prompt ordering, or parametric knowledge override, then outline systematic debugging. Do not just say hallucination.
Describe a basic RAG architecture and its two main components
This tests retrieval-generation separation. Good answers name the retriever, which fetches relevant documents, and the generator, which synthesizes an answer using those documents plus the query.
Evaluating RAG Systems: Metrics for Retrieval and Generation
Evaluating a RAG system means grading its two parts: retrieval and generation. Metrics like relevance check if the right documents were found, while faithfulness and accuracy check if the final answer correctly uses those documents.
Document Chunking: Slicing Text for LLMs
Think of chunking as preparing text "bites" for an LLM. It breaks large documents into smaller, meaningful segments to fit a model's context window and improve search. It's essential for Retrieval-Augmented Generation (RAG) and semantic search.
LLM Agents: Giving Models Tools and a Plan
An LLM Agent gives a model tools and a plan to solve complex problems. Instead of just answering a question, it breaks it down, uses APIs or code interpreters, and remembers past steps. The footgun is thinking RAG is an agent; agents need planning and.
Graph RAG: Answering Questions with Connected Facts
Graph RAG answers complex questions by exploring a map of connected facts (a knowledge graph) instead of just searching flat text. Use it for queries needing synthesis, like finding drugs for a disease made by companies in a specific country.
Self-Querying Retriever: Let an LLM Write Its Own Filters
A self-querying retriever uses an LLM to turn a natural language question into a structured query with metadata filters. It lets users ask things like "Find documents about Python from before 2020," which a simple vector search can't do.
Context Stuffing: Giving LLMs Short-Term Memory
Context stuffing adds external documents to an LLM's prompt, giving it temporary, task-specific knowledge. Use it for one-off questions on specific docs, but beware: it fails when documents exceed the model's context window limit, causing truncated data.
Hybrid Search: Combining Keyword and Vector Search
Hybrid search combines keyword precision with vector search's conceptual understanding in one query. It excels at retrieving relevant documents for RAG by finding both exact matches (like names) and similar ideas.
Dense Passage Retrieval (DPR): Semantic Search for QA
DPR finds answers by meaning, not just keywords. It converts questions and documents into vectors and finds the closest matches, forming the core of Retrieval-Augmented Generation (RAG).
Embedding Models: The 'Retrieval' in RAG
An embedding model acts like a librarian for your data, converting text into numerical vectors so similar concepts are grouped together. In RAG, it finds relevant documents to feed an LLM, but using the wrong model type will yield poor retrieval results.
Extrinsic vs. In-Context: Two Types of LLM Hallucination
LLM hallucinations split into two types: in-context, where output contradicts provided sources, and extrinsic, where it conflicts with world knowledge. This distinction is critical for engineers debugging AI systems, as RAG pipelines fight in-context errors while open-ended generation faces extrinsic ones. Mitigating extrinsic hallucinations requires models to not only be factual but also to admit when they don't know an answer, a major challenge given the impracticality of verifying against tra
Get RAG bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.