RAG vs. Fine-Tuning: Key Differences

### What this tests Your understanding of the architectural and practical differences between the two main strategies for specializing a foundation model. It's a test of knowing *when* and *why* to apply each technique based on cost, maintenance, and the nature of the task (knowledge vs. skill).

### A good answer covers * **Knowledge Source & Timing:** Fine-tuning encodes knowledge into the model's *parametric memory* (weights) during an offline training process. RAG uses an external *non-parametric memory* (a vector database of documents) and retrieves information at *inference time*. * **Updating Knowledge:** RAG is far easier to keep current. You simply update the document database without touching the model, making it ideal for dynamic information. Fine-tuning requires a costly and time-consuming retraining process to incorporate new knowledge. * **Explainability & Hallucination:** RAG provides provenance by citing the retrieved documents, making its outputs more verifiable and reducing factual hallucinations. Fine-tuned models cannot easily trace an assertion back to a specific source document. * **Primary Use Case:** Fine-tuning is best for teaching a model a new *skill, style, or format* (e.g., 'always respond in JSON', 'adopt this persona'). RAG is best for *knowledge-intensive tasks* requiring up-to-date, factual information (e.g., 'what were our Q3 sales figures?'). * **Cost & Latency:** Fine-tuning is computationally expensive upfront but has low inference latency. RAG has a higher inference-time latency and cost (due to the retrieval step) but is vastly cheaper for ongoing knowledge updates.

### Common wrong answers * **"RAG is just better."** This ignores that fine-tuning is superior for teaching skills, styles, and behaviors where the knowledge isn't in a document. * **Treating them as mutually exclusive.** Advanced workflows often combine them: fine-tune a model on a domain's style and terminology, then use RAG to inject real-time data. * **Vague descriptions like "RAG adds documents."** A senior answer specifies the retrieve-and-generate pipeline, including the vector index and retriever components.

### Follow-up the interviewer might ask * "When would you use both together? Walk me through that architecture." * "Let's say we need to build a chatbot for internal company documents that are updated daily. Which would you choose and what are the cost implications?"

### One concrete example "To build a customer support bot, I'd first fine-tune a model on 5,000 high-quality support conversation transcripts to teach it the company's tone and conversational style. Then, I would implement RAG on our entire knowledge base of 100,000+ technical documents so the fine-tuned bot can retrieve and cite specific, up-to-date product specifications at runtime to answer user questions."