When to choose RAG over fine-tuning

WHAT THIS TESTS This checks whether you understand the orthogonal roles of the two techniques: RAG supplies knowledge at inference time, fine-tuning shapes behavior in the weights, and that they are often complementary rather than competing.

A GOOD ANSWER COVERS Choose RAG when the problem is knowledge: a large corpus, facts that change frequently, or a need to cite sources and reduce hallucination. RAG retrieves relevant passages at query time and conditions the answer on them, so updating knowledge means re-indexing documents, which is fast and cheap, and answers can point to their sources. Choose fine-tuning when the problem is behavior: a consistent output format, a domain tone or style, following a specialized instruction pattern, or learning a task the base model performs poorly. Fine-tuning bakes these into the weights so they apply without retrieval overhead. The trade-offs: RAG adds retrieval latency and depends on retrieval quality, and a bad retriever yields bad answers, but it is easy to keep current and transparent. Fine-tuning has no retrieval step at inference but is expensive to retrain, risks catastrophic forgetting, cannot keep volatile facts fresh, and is opaque. In practice teams combine them: fine-tune for format and domain behavior, and use RAG for current factual grounding.

COMMON WRONG ANSWERS Using fine-tuning to inject frequently changing facts; claiming RAG teaches the model new skills or styles; treating the two as mutually exclusive; ignoring retrieval quality as RAG's weak point; forgetting catastrophic forgetting in fine-tuning.

LIKELY FOLLOW-UPS When do you combine both? How does retrieval quality bound RAG performance? What is catastrophic forgetting? How do you keep a fine-tuned model's facts from going stale?

ONE CONCRETE EXAMPLE A support assistant must answer from a knowledge base updated weekly and cite articles, so RAG fits the factual, changing, citable need. Separately, the company wants every answer in a specific structured format with a friendly brand voice, so they lightly fine-tune the model for that format and tone while RAG supplies the current facts.