Top 30 Fine Tuning Interview Questions and Answers

30 questions. Pick an answer, or open “Show the answer” to read it.

Answers are graded in your browser. Nothing is saved, and no XP or streak is earned here. The app keeps score.

1. Question 1 of 30

   When is a learning rate schedule particularly crucial for achieving state-of-the-art results?

   • aWhen the model consistently overfits the training data during initial epochs.
   • bWhen training simple models where a fixed learning rate is insufficient.
   • cWhen using optimizers that already manage learning rates adaptively, like AdaFactor.
   • dWhen fine-tuning large, complex models such as transformers.
   Show the answer

   Answer: d · When fine-tuning large, complex models such as transformers.

   C is correct because the card states schedules are "standard practice for training and fine-tuning large models, especially transformers... essential for achieving state-of-the-art results." B is a tempting distractor, but the card explicitly warns against using external schedules with optimizers like AdaFactor due to potential conflicts.

   Read the full bite: Learning Rate Scheduling: A Gearbox for Model Training

2. Question 2 of 30

   Which statement best describes how supervised fine-tuning updates a pre-trained language model?

   • aIt trains on raw internet text using the same objective as pre-training but with a larger learning rate.
   • bIt updates existing weights by predicting the next token only on curated assistant completions, masking the prompt.
   • cIt switches the training objective from causal language modeling to reward maximization.
   • dIt freezes all base parameters and automatically adds new trainable layers to the model architecture.
   Show the answer

   Answer: b · It updates existing weights by predicting the next token only on curated assistant completions, masking the prompt.

   Supervised fine-tuning updates existing weights by continuing next-token prediction on curated prompt-completion data, computing loss only on assistant tokens while masking prompts. The most tempting distractor is wrong because SFT deliberately uses curated demonstrations and a small learning rate, not raw internet text like pre-training.

   Read the full bite: Describe supervised fine-tuning for a pre-trained language model

3. Question 3 of 30

   A team wants their LLM to reliably cite this week's internal sales figures. Why is fine-tuning the base model a poor primary tool for this requirement?

   • aFine-tuning can only be done on unlabeled data, so facts cannot be added
   • bFine-tuning requires more compute than the original pre-training run
   • cFine-tuning mainly steers existing capabilities and is unreliable for injecting fresh, specific facts
   • dFine-tuning always erases the model's grammar and language ability
   Show the answer

   Answer: c · Fine-tuning mainly steers existing capabilities and is unreliable for injecting fresh, specific facts

   Fine-tuning adapts and steers learned behaviors rather than dependably storing new facts, so retrieval is better for fresh data. It does not cost more than pre-training, nor does it wipe out language ability.

   Read the full bite: Pre-training versus fine-tuning an LLM

4. Question 4 of 30

   When adapting a large model to dozens of tasks on a tight compute budget, why is LoRA strongly preferred over full fine-tuning?

   • aIt trains only a small number of extra parameters per task, keeping one shared base model and sharply reducing compute, storage, and serving costs while matching performance.
   • bFull fine-tuning always outperforms LoRA, so the increased cost is justified when adapting to many tasks.
   • cLoRA primarily lowers training FLOPs but has the same storage and inference requirements as full fine-tuning because each task still needs its own complete checkpoint.
   • dYou can average full fine-tuned weights across tasks to avoid storing multiple checkpoints, making full fine-tuning as storage-efficient as LoRA.
   Show the answer

   Answer: a · It trains only a small number of extra parameters per task, keeping one shared base model and sharply reducing compute, storage, and serving costs while matching performance.

   LoRA adds lightweight adapters per task around a single frozen base model, drastically cutting compute, storage, and inference costs while maintaining comparable performance. Distractor B is wrong because the card explicitly flags the belief that full fine-tuning always outperforms PEFT as a major red flag, and storing separate full checkpoints for many tasks is prohibitively expensive.

   Read the full bite: Full fine-tuning or LoRA on a tight compute budget?

5. Question 5 of 30

   Why does LoRA reduce GPU memory usage during fine-tuning compared to full fine-tuning?

   • aIt quantizes the frozen base weights to lower precision, shrinking the model footprint.
   • bIt freezes the original weight matrix W and trains only low-rank matrices A and B, so gradients and optimizer states are only stored for A and B.
   • cIt reduces the rank of the pretrained weight matrix W itself, lowering its memory footprint.
   • dIt merges the low-rank update into W during each training step, reducing active parameters.
   Show the answer

   Answer: b · It freezes the original weight matrix W and trains only low-rank matrices A and B, so gradients and optimizer states are only stored for A and B.

   LoRA keeps the original weight matrix W frozen and only trains the small low-rank matrices A and B, which means gradients and optimizer states like those in Adam are only maintained for A and B, drastically reducing memory usage. The most tempting distractor claims the update is merged during training, but merging BA into W actually happens after training to preserve inference speed, not to save memory during fine-tuning.

   Read the full bite: How does LoRA work and why is it memory-efficient?

6. Question 6 of 30

   When an LLM regresses on legacy tasks after domain-specific fine-tuning, which approach best balances new task mastery with preserving existing capabilities?

   • aFreeze the first few transformer layers and fully fine-tune all remaining parameters
   • bLower the learning rate so updates are too small to overwrite prior knowledge
   • cAdopt LoRA with a regularization term that penalizes deviation from base weights while evaluating on holdout legacy tasks
   • dTreat the regression as overfitting and expand the new domain training set
   Show the answer

   Answer: c · Adopt LoRA with a regularization term that penalizes deviation from base weights while evaluating on holdout legacy tasks

   LoRA restricts the update space, regularization prevents drift from base capabilities, and legacy evaluation detects regression early. Simply lowering the learning rate is a common misconception because gradients can still overwrite critical knowledge even at small step sizes.

   Read the full bite: What is catastrophic forgetting in LLMs and how do you mitigate it?

7. Question 7 of 30

   What is the central mathematical idea that lets DPO align a model without a separate reward model?

   • aA reward model is still trained but discarded before PPO begins
   • bPreference labels can be ignored because DPO only trains on chosen responses
   • cIt replaces gradient descent with reinforcement learning rollouts
   • dThe optimal KL-constrained policy lets the reward be re-expressed via the policy and reference probabilities
   Show the answer

   Answer: d · The optimal KL-constrained policy lets the reward be re-expressed via the policy and reference probabilities

   DPO exploits the closed-form optimal policy of the KL-constrained objective to fold the reward into the policy-reference probability ratio, removing the reward network. It still uses rejected responses and standard gradient descent.

   Read the full bite: Direct Preference Optimization explained

8. Question 8 of 30

   What is the primary objective of Supervised Fine-Tuning (SFT) for a pre-trained base Large Language Model (LLM)?

   • aTo enhance the model's ability to generate more creative and imaginative text.
   • bTo improve the model's speed and efficiency in processing large volumes of text.
   • cTo adapt the model to follow instructions and engage in helpful, conversational interactions.
   • dTo teach the model new factual information about specific domains or topics.
   Show the answer

   Answer: c · To adapt the model to follow instructions and engage in helpful, conversational interactions.

   The card explains that SFT exists to transform a knowledgeable but unhelpful base model into a 'helpful, instruction-following assistant' by teaching it conversational skills. It explicitly states that SFT is not for teaching new domain-specific knowledge.

   Read the full bite: Supervised Fine-Tuning (SFT): Teaching a Model to Chat

9. Question 9 of 30

   What is the fundamental reason catastrophic forgetting occurs in neural networks?

   • aThe model's internal parameters, optimized for new tasks, overwrite the configurations essential for previous knowledge.
   • bNeural networks have a fixed memory capacity, and new information displaces older data.
   • cAI models are designed to selectively discard information deemed less relevant when fine-tuned on specialized datasets.
   • dThe new training data is inherently more complex, causing the model to prioritize learning it over simpler, previously acquired skills.
   Show the answer

   Answer: a · The model's internal parameters, optimized for new tasks, overwrite the configurations essential for previous knowledge.

   The card explains that catastrophic forgetting happens because neural networks adjust shared internal parameters (weights) to optimize for new tasks, which inadvertently overwrites the configurations vital for previously learned information. It is not due to a fixed memory capacity, but rather the process of parameter adjustment.

   Read the full bite: Catastrophic Forgetting: The AI's Amnesia Problem

10. Question 10 of 30

    A developer needs to adapt a large foundation model for a specific task using limited computational resources. Which method is most appropriate?

    • aParameter-Efficient Fine-Tuning (PEFT)
    • bFull fine-tuning of all model parameters
    • cUsing the foundation model as-is without any modifications
    • dTraining a new, smaller model from scratch
    Show the answer

    Answer: a · Parameter-Efficient Fine-Tuning (PEFT)

    PEFT is specifically designed to adapt large models for specific tasks on a budget by freezing most of the original model and only training a small number of new parameters. Full fine-tuning (B) is too resource-intensive for limited hardware, and training a new model from scratch (D) is even more demanding and inefficient for adapting an existing foundation model.

    Read the full bite: PEFT: Fine-Tune Large Models on a Budget

11. Question 11 of 30

    Which statement accurately describes a key outcome of instruction fine-tuning for a base LLM?

    • aIt drastically reduces the computational resources required for the model's deployment and operation.
    • bIt fundamentally alters the model's core architecture to improve its reasoning capabilities.
    • cIt primarily enhances the model's ability to acquire and retain new factual knowledge.
    • dIt teaches the model to generate responses that align with user intent and specific task formats.
    Show the answer

    Answer: d · It teaches the model to generate responses that align with user intent and specific task formats.

    Instruction fine-tuning is designed to align the model's behavior with user intent, teaching it to follow directions and produce compliant, helpful responses in specific formats. The card explicitly states that it does not inject new factual knowledge; rather, it teaches conversational style and format.

    Read the full bite: Instruction Fine-Tuning: Teaching LLMs to Follow Orders

12. Question 12 of 30

    For which application scenario are adapter modules most beneficial for fine-tuning a large language model?

    • aAdapting a single base model to numerous distinct downstream tasks with limited storage and compute.
    • bAchieving the absolute highest performance on a single, highly specialized task.
    • cSignificantly altering the foundational knowledge of the base model for a completely new domain.
    • dWhen the target task is extremely complex and requires extensive changes to the model's core architecture.
    Show the answer

    Answer: a · Adapting a single base model to numerous distinct downstream tasks with limited storage and compute.

    Adapter modules are designed for efficiently adapting a single base model to multiple tasks, saving resources by only storing small, task-specific weights. While tempting, aiming for absolute maximum performance on a single, critical task might still favor full fine-tuning if resources are unlimited, as adapters might not match its performance in such cases.

    Read the full bite: Adapter Modules: Efficient LLM Fine-Tuning

13. Question 13 of 30

    What is the primary advantage of using transfer learning for computer vision tasks when your specific dataset is small?

    • aIt simplifies the neural network architecture, making it easier to train on limited resources.
    • bIt automatically augments the small dataset with synthetic images, increasing its size.
    • cIt allows the model to be trained from scratch on a smaller dataset, which is faster.
    • dIt leverages general features learned from a large, diverse dataset, reducing the need for extensive specific data.
    Show the answer

    Answer: d · It leverages general features learned from a large, diverse dataset, reducing the need for extensive specific data.

    Transfer learning's main advantage is reusing foundational knowledge (general features like edges and textures) learned from a massive dataset, meaning the model needs less new data to adapt to specific patterns. Option C is incorrect because transfer learning explicitly avoids training from scratch.

    Read the full bite: Transfer Learning: Don't Train Vision Models from Scratch

14. Question 14 of 30

    In an LLM ad copy system with human reviewers, how should editor rejections and modifications be handled to maximize long-term model improvement?

    • aUse them to prompt-engineer the base model in real time before the next generation batch.
    • bStore them solely for compliance auditing without feeding back into the training pipeline.
    • cDiscard them after review to reduce storage overhead, since only approved copy reflects brand standards.
    • dConvert them into preference pairs for reward modeling and retain approved copy for future supervised fine-tuning.
    Show the answer

    Answer: d · Convert them into preference pairs for reward modeling and retain approved copy for future supervised fine-tuning.

    Human rejections and edits should become preference pairs for RLHF or reward model updates, while approved copy is added to the golden dataset for future supervised fine-tuning. Storing feedback only for compliance audits is a common anti-pattern that treats human review as a static gate rather than a continuous training signal.

    Read the full bite: Design an LLM ad copy system with human-in-the-loop

15. Question 15 of 30

    What core problem does RLHF address that standard next-token pretraining cannot?

    • aEliminating the need for any human-labeled data during training
    • bGuaranteeing the model never produces factually incorrect statements
    • cAligning outputs with human intent like helpfulness and harmlessness, which the likelihood objective never encodes
    • dReducing the parameter count of the model to fit on smaller GPUs
    Show the answer

    Answer: c · Aligning outputs with human intent like helpfulness and harmlessness, which the likelihood objective never encodes

    Pretraining maximizes text likelihood, not human values, so RLHF aligns behavior with preferences. RLHF needs human preference data, does not shrink models, and shapes preferences rather than guaranteeing factual truth.

    Read the full bite: What RLHF is and the safety problem it solves

16. Question 16 of 30

    When fine-tuning BERT on a niche dataset of only 800 labeled examples, which approach best preserves pretrained knowledge while avoiding overfitting?

    • aTrain BERT from scratch on the 800 examples so it learns domain-specific patterns directly
    • bUse a learning rate of 1e-3 on all layers to fully adapt the pretrained weights to the new domain
    • cFeed raw untokenized text into the model and rely on the CLS token without an attention mask
    • dStart from a domain-adjacent checkpoint, freeze the first eight layers, and fine-tune at 2e-5 with early stopping
    Show the answer

    Answer: d · Start from a domain-adjacent checkpoint, freeze the first eight layers, and fine-tune at 2e-5 with early stopping

    Starting from a domain-adjacent checkpoint and freezing early layers preserves low-level features while a tiny learning rate and early stopping prevent overfitting on scarce labels. Training from scratch is infeasible with only 800 examples, a large learning rate destroys pretrained weights, and raw text without tokenization or attention masks is incompatible with the model's input requirements.

    Read the full bite: How do you leverage and fine-tune BERT for niche classification?

17. Question 17 of 30

    In DP-SGD, what is the relationship between the epsilon parameter and the strength of the privacy guarantee?

    • aA smaller epsilon gives a stronger privacy guarantee but typically lowers utility
    • bEpsilon controls only training speed, not privacy
    • cEpsilon must equal delta for the guarantee to hold
    • dA larger epsilon gives a stronger privacy guarantee
    Show the answer

    Answer: a · A smaller epsilon gives a stronger privacy guarantee but typically lowers utility

    Smaller epsilon bounds privacy loss more tightly, requiring more noise and usually reducing accuracy. Larger epsilon is a weaker guarantee, so the first option reverses the relationship.

    Read the full bite: Differential privacy vs utility in LLM fine-tuning

18. Question 18 of 30

    Why is RAG generally preferred over fine-tuning when the underlying knowledge base changes every day?

    • aFine-tuning permanently removes hallucinations
    • bRAG lets you update knowledge by re-indexing instead of retraining weights
    • cRAG always produces shorter answers
    • dFine-tuning cannot run on GPUs
    Show the answer

    Answer: b · RAG lets you update knowledge by re-indexing instead of retraining weights

    RAG decouples knowledge from weights, so updates mean re-indexing documents. Fine-tuning bakes facts into weights, so daily-changing knowledge would require constant, costly retraining.

    Read the full bite: Fine-tuning vs RAG for daily-updated docs

19. Question 19 of 30

    With only a small labeled dataset, what is the safest way to adapt a pretrained ResNet50?

    • aRetrain all layers from scratch at a high learning rate
    • bFine-tune only the earliest convolutional layers
    • cKeep the original 1000-class head and add a separate classifier on top
    • dFreeze the backbone, replace the head, train it, then optionally fine-tune top blocks at a low rate
    Show the answer

    Answer: d · Freeze the backbone, replace the head, train it, then optionally fine-tune top blocks at a low rate

    Freezing the generic backbone and training a new head avoids overfitting on small data, with cautious top-layer fine-tuning as an optional gain. Retraining everything overfits, and the early layers hold the most transferable features, so they should stay frozen.

    Read the full bite: Transfer learning from ResNet50 on small data

20. Question 20 of 30

    When fine-tuning a pretrained model with the transformers library, why must the tokenizer be loaded from the same checkpoint as the model?

    • aBecause datasets cannot work without it
    • bSo the vocabulary, token IDs, and special tokens align with what the model was trained on
    • cTokenizers are interchangeable, so it does not matter
    • dTo make training run on a GPU
    Show the answer

    Answer: b · So the vocabulary, token IDs, and special tokens align with what the model was trained on

    Each model expects a specific vocabulary and special-token scheme. A mismatched tokenizer produces token IDs the model never saw, corrupting inputs. Tokenizers are not interchangeable across models.

    Read the full bite: Hugging Face Hub, transformers, and datasets

21. Question 21 of 30

    Which situation most clearly favors RAG over fine-tuning?

    • aYou need the model to always reply in a fixed JSON structure
    • bYou need answers grounded in a frequently updated, citable knowledge base
    • cYou need the model to adopt a specific brand tone of voice
    • dYou need to teach the model a new specialized task it performs poorly
    Show the answer

    Answer: b · You need answers grounded in a frequently updated, citable knowledge base

    RAG injects fresh, citable knowledge at query time and updates by re-indexing, ideal for changing facts. Fixed formatting, tone, and learning new task behavior are jobs for fine-tuning, which shapes the weights.

    Read the full bite: When to choose RAG over fine-tuning

22. Question 22 of 30

    What design choice is essential for an evaluation suite meant to detect catastrophic forgetting across continual fine-tuning rounds?

    • aThe benchmark should be regenerated from the latest training data each round
    • bOnly the aggregate score on new tasks needs tracking
    • cThe suite should be discarded once the model passes once
    • dA frozen, held-out benchmark of original capabilities run after every update for comparable deltas
    Show the answer

    Answer: d · A frozen, held-out benchmark of original capabilities run after every update for comparable deltas

    A stable, leak-free benchmark of prior capabilities makes scores comparable over time so regressions are visible. Regenerating it from new data or tracking only new-task scores would hide the very forgetting you want to catch.

    Read the full bite: Detecting catastrophic forgetting in continual fine-tuning

23. Question 23 of 30

    Why is counterfactual data augmentation (swapping demographic markers while holding intent constant) effective for reducing name-correlated bias in a fine-tuned chatbot?

    • aIt teaches the model that the demographic marker should not change response quality by presenting otherwise-identical examples
    • bIt removes all names from the training data so the model never sees demographic signals
    • cIt replaces fine-tuning entirely so no bias can be learned
    • dIt increases the total dataset size, which always improves fairness
    Show the answer

    Answer: a · It teaches the model that the demographic marker should not change response quality by presenting otherwise-identical examples

    Augmentation pairs identical intents across demographic markers, forcing the model to treat the marker as irrelevant to quality. Simply adding data volume does not target the disparity and can even amplify it.

    Read the full bite: Mitigating demographic bias in a fine-tuned chatbot

24. Question 24 of 30

    When scrubbing conversational data for fine-tuning, why does detection recall typically matter more than precision?

    • aA single missed identifier can be memorized and regurgitated by the model, while over-redaction mainly costs some utility
    • bRecall is cheaper to compute than precision
    • cPrecision is impossible to measure on text data
    • dHigher precision always lowers recall, so recall is the only tunable metric
    Show the answer

    Answer: a · A single missed identifier can be memorized and regurgitated by the model, while over-redaction mainly costs some utility

    Leaked PII can be memorized and leaked at inference, a serious privacy harm, so missing identifiers (low recall) is worse than occasionally over-redacting. Precision and recall are both measurable and not strictly inverse.

    Read the full bite: Preprocessing conversations to protect privacy before fine-tuning

25. Question 25 of 30

    What is the primary advantage of model merging over traditional fine-tuning for combining LLM capabilities?

    • aIt allows combining models trained on vastly different base architectures without compatibility issues.
    • bIt offers a computationally inexpensive way to combine complementary specialized capabilities post-training.
    • cIt is primarily used to create entirely new, emergent capabilities not present in the original models.
    • dIt ensures the merged model will always retain the full performance of its best parent model.
    Show the answer

    Answer: b · It offers a computationally inexpensive way to combine complementary specialized capabilities post-training.

    Model merging is presented as a "cheap, post-training method" to combine specialized capabilities, directly addressing the computational expense of fine-tuning for every niche. Option A is incorrect because the card explicitly states not to merge models with different underlying architectures.

    Read the full bite: Model Merging: Combine LLM Skills Without Retraining

26. Question 26 of 30

    Under which circumstance is full fine-tuning the most suitable approach for adapting a pre-trained model?

    • aTo quickly adapt a pre-trained model to a new, small dataset without significant computational cost.
    • bWhen the main objective is to prevent the model from forgetting its original general knowledge.
    • cAs a lightweight alternative to Parameter-Efficient Fine-Tuning (PEFT) for minor task adjustments.
    • dTo achieve the highest possible performance on a specific, complex domain, assuming ample computational resources.
    Show the answer

    Answer: d · To achieve the highest possible performance on a specific, complex domain, assuming ample computational resources.

    Full fine-tuning is recommended when maximum performance on a highly specialized domain is critical and significant computational resources are available, as it deeply embeds new knowledge by updating all model parameters. It is explicitly noted as prone to catastrophic forgetting and is not a lightweight or low-cost option, making other choices incorrect.

    Read the full bite: Full Fine-Tuning: Updating Every Model Parameter

27. Question 27 of 30

    Which scenario most compellingly justifies choosing LoRA over full fine-tuning for adapting a large language model?

    • aA. The objective is to achieve the highest possible performance on a single, highly specialized task, with no budget constraints.
    • bB. The need is to quickly develop and deploy multiple distinct task-specific models, each requiring minimal GPU memory and storage for efficient management.
    • cC. The adaptation requires fundamentally altering the base model's deep internal representations to handle a completely novel data distribution.
    • dD. The critical constraint is that the fine-tuned model must have absolutely no increase in inference latency compared to the base model, even if it involves a more complex training pipeline.
    Show the answer

    Answer: b · B. The need is to quickly develop and deploy multiple distinct task-specific models, each requiring minimal GPU memory and storage for efficient management.

    Option B correctly identifies LoRA's strength in enabling efficient, rapid development and deployment of multiple specialized models under resource constraints. While LoRA does not add inference latency (Option D), its primary justification often lies in its significant resource savings and ability to manage many adaptations, and its training pipeline is simpler, not more complex.

    Read the full bite: When would you use LoRA vs full fine-tuning?

28. Question 28 of 30

    Which statement accurately describes a unique advantage of LoRA's fine-tuning approach during model inference?

    • aA. Specialized models maintain the identical inference speed of the original base model due to weight merging.
    • bB. Inference becomes faster because only the smaller A and B matrices are processed during the forward pass.
    • cC. It reduces the memory footprint of the active model during inference by only loading necessary A and B weights on demand.
    • dD. It introduces new, optimized layers that run in parallel with the base model, speeding up overall computation.
    Show the answer

    Answer: a · A. Specialized models maintain the identical inference speed of the original base model due to weight merging.

    The card states that LoRA's key benefit is that the A and B matrices can be merged into the original weights (W' = W + BA), meaning the adapted model has the exact same size and inference speed as the original. Option B is incorrect because the merged W' matrix, which is the same size as W, is processed, not just A and B separately.

    Read the full bite: LoRA: Fine-Tuning LLMs with a Fraction of the Cost

29. Question 29 of 30

    Which statement best describes how Direct Preference Optimization (DPO) fundamentally differs from traditional Reinforcement Learning from Human Feedback (RLHF) for aligning LLMs?

    • aDPO directly fine-tunes the language model using a classification loss on preference pairs, eliminating the need for a separate reward model.
    • bDPO employs a more advanced reinforcement learning algorithm that is inherently more stable and efficient than PPO.
    • cDPO first trains a highly accurate reward model and then uses it to generate synthetic data for supervised fine-tuning.
    • dDPO focuses on pre-training a stronger base model that inherently understands human preferences, reducing the need for post-training alignment.
    Show the answer

    Answer: a · DPO directly fine-tunes the language model using a classification loss on preference pairs, eliminating the need for a separate reward model.

    DPO's core innovation is treating the LLM as its own reward model, directly adjusting its parameters based on human preference pairs using a classification loss, thereby bypassing the need to train a separate reward model and the complexities of reinforcement learning. Option B is incorrect because DPO avoids reinforcement learning entirely, opting for a direct fine-tuning approach.

    Read the full bite: Direct Preference Optimization (DPO): Your LLM is a Reward Model

30. Question 30 of 30

    For an LLM application that needs to incorporate rapidly changing factual information and provide verifiable sources, which approach is most suitable?

    • aA) Retrieval-Augmented Generation (RAG)
    • bB) Fine-tuning the LLM on new data
    • cC) Increasing the LLM's context window
    • dD) Re-pretraining the LLM with the updated dataset
    Show the answer

    Answer: a · A) Retrieval-Augmented Generation (RAG)

    RAG is ideal for dynamic information, allowing easy updates to an external knowledge base without retraining the model, and provides verifiable sources through citations. Fine-tuning, while incorporating new data, requires costly retraining for updates and does not inherently offer source traceability, making it less suitable for rapidly changing information or verifiable sources.

    Read the full bite: RAG vs. Fine-Tuning: Key Differences