Fine Tuning
30 bites tagged Fine Tuning — interview questions with model answers, and 60-second explainers.
Preprocessing conversations to protect privacy before fine-tuning
Detect and redact PII with NER plus regex, choose redaction versus pseudonymization, and validate recall. Privacy-preserving data pipelines for training.
Mitigating demographic bias in a fine-tuned chatbot
Curate or counterfactually augment training data to balance demographics, plus apply post-hoc guardrails or fairness-constrained fine-tuning. Practical bias mitigation across the ML lifecycle.
Detecting catastrophic forgetting in continual fine-tuning
Maintain a frozen held-out benchmark of original capabilities, evaluate after every fine-tune, track per-capability deltas, and alert on regressions. guarding original skills during continual training.
When to choose RAG over fine-tuning
RAG for fresh, factual, citable knowledge that changes often, fine-tuning for behavior, style, or format the model must internalize. matching technique to the kind of adaptation needed.
Hugging Face Hub, transformers, and datasets
The Hub hosts models and data, transformers loads models and tokenizers and provides the Trainer, datasets streams and maps preprocessing. practical fluency with the standard NLP toolchain.
Fine-tuning vs RAG for daily-updated docs
Choose RAG because docs change daily, embed and index chunks in a vector store, retrieve top matches and inject into the prompt. matching the right technique to freshness needs.
Differential privacy vs utility in LLM fine-tuning
Clipping plus calibrated noise per step, smaller epsilon means stronger privacy but degraded accuracy, tracking the privacy budget across epochs. understanding DP-SGD's noise-for-privacy bargain.
What RLHF is and the safety problem it solves
Collect human preference rankings, train a reward model, fine-tune the policy with PPO; it aligns outputs with human intent the loss function cannot specify. grasp of alignment via RLHF.
Direct Preference Optimization explained
DPO reparameterizes the RLHF reward in terms of the policy itself, turning alignment into a simple classification loss on preference pairs with no separate reward model or PPO. understanding of DPO versus RLHF.
Pre-training versus fine-tuning an LLM
Pre-training is broad self-supervised next-token prediction on huge corpora at massive cost; fine-tuning adapts on small labeled data cheaply. grasp of the two-stage LLM training lifecycle.
Transfer learning from ResNet50 on small data
Replace the final classification head with one sized to your classes, freeze the pretrained convolutional backbone as a feature extractor, train the new head, then optionally fine-tune top blocks at a low… applying transfer learning.
What is catastrophic forgetting in LLMs and how do you mitigate it?
This tests stability-plasticity trade-offs in fine-tuning. A strong answer defines catastrophic forgetting as lost prior capabilities, cites LoRA, regularization, and continual learning.
How does LoRA work and why is it memory-efficient?
LoRA freezes weights and trains A and B so delta-W equals BA, cutting trainable params 10,000x and memory 3x since only A and B get grads. Low-rank adaptation. Claiming it shrinks size or adds latency.
Full fine-tuning or LoRA on a tight compute budget?
This tests budget-constrained adaptation for many tasks. A strong answer picks LoRA: it trains only a small number of extra parameters, cutting compute and storage versus full fine-tuning while matching performance.
Describe supervised fine-tuning for a pre-trained language model
Tests if you know SFT aligns a base model to instructions using curated prompt-completion data. A strong answer covers next-token prediction on completions, conversational formats, and small learning rates.
How do you leverage and fine-tune BERT for niche classification?
Tests transfer learning with scarce labels. Outline: pick a domain-adjacent checkpoint, add a classification head, use learning rates near 2e-5 with early stopping, and stratify tiny validation splits.
Design an LLM ad copy system with human-in-the-loop
LoRA on approved copy, inference guardrails, human review, feedback as preference pairs for RLHF. Architecture for fine-tuning, guardrails, and human feedback loops. Treating review as static gate, not training signal.
Full Fine-Tuning: Updating Every Model Parameter
Full fine-tuning updates all weights of a pre-trained model on your new data, unlike methods that only change a small fraction. Use it to deeply embed new knowledge, but beware: it's costly and risks making the model forget its original general skills.
Model Merging: Combine LLM Skills Without Retraining
Model merging blends specialized LLMs into one, like creating a custom alloy from different metals. It's used to combine a coding expert with a legal expert, for example, without costly retraining.
Adapter Modules: Efficient LLM Fine-Tuning
Adapters are small modules plugged into a frozen LLM to avoid costly full fine-tuning. This lets you specialize a base model for many tasks by training tiny, swappable plugins instead of duplicating the entire model for each task.
Instruction Fine-Tuning: Teaching LLMs to Follow Orders
Instruction fine-tuning teaches a base LLM to follow commands, not just predict the next word. It turns a raw text-completion engine into a helpful assistant, enabling it to answer questions or summarize text. The footgun: it learns style, not facts.
PEFT: Fine-Tune Large Models on a Budget
Parameter-Efficient Fine-Tuning (PEFT) adapts huge models without retraining everything. It's like adding a task-specific cheat sheet to a genius brain. Use it to specialize LLMs on consumer GPUs.
Catastrophic Forgetting: The AI's Amnesia Problem
Catastrophic forgetting is when an AI, trained on new information, abruptly forgets what it previously knew. It's like overwriting a file instead of appending to it. This happens when fine-tuning a model on a narrow dataset, erasing its general knowledge.
Supervised Fine-Tuning (SFT): Teaching a Model to Chat
Supervised Fine-Tuning (SFT) teaches a general LLM to be a helpful assistant by training it on high-quality conversations. This turns a base model into an instruction-following chatbot.
Get Fine Tuning bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.