Describe supervised fine-tuning for a pre-trained language model
Tests if you know SFT aligns a base model to instructions using curated prompt-completion data. A strong answer covers next-token prediction on completions, conversational formats, and small learning rates.
What's really being asked
This question checks whether you understand supervised fine-tuning as the critical post-training alignment step that turns a general pre-trained base model into an instruction-following assistant. Interviewers want to see that you know SFT requires curated demonstration data, preserves the causal language modeling objective, and differs fundamentally from pre-training on raw corpora.
The full answer
First, define SFT as continued training on a smaller, high-quality dataset of prompt-response pairs where the model learns to emulate desired outputs. Second, describe the dataset types: standard prompt-completion pairs or conversational message lists with user and assistant roles. Third, state the primary objective remains next-token prediction, but loss is typically computed only on the completion or assistant tokens while prompt tokens are masked. Fourth, note practical details such as using a small learning rate, training for only a few epochs, and optionally applying chat templates to format conversations correctly.
The mistakes people make
Do not say SFT uses raw unlabeled internet text like pre-training; it relies on curated demonstrations. Do not claim the objective switches to classification, regression, or reward maximization; it stays causal LM. Avoid stating that both prompts and completions contribute equally to the loss; best practice masks the prompt to avoid wasting gradient updates on inputs. Do not suggest SFT automatically adds new parameters; standard full fine-tuning updates existing weights, while parameter-efficient methods like LoRA are separate choices.
What usually comes next
Expect questions on how SFT differs from RLHF or preference optimization methods like DPO. Interviewers may ask why prompt masking matters, how to choose between full fine-tuning and LoRA, what hyperparameters prevent catastrophic forgetting, or how multi-turn conversations are formatted and truncated.
A concrete example
Using Hugging Face TRL, you might load a pre-trained Qwen 0.6B model and a conversational dataset such as Capybara. You format each example as a messages list with user and assistant roles. The SFTTrainer automatically applies the model's chat template and computes next-token prediction loss only on the assistant content. You train with a small learning rate for one to three epochs, producing a model that follows instructions in the style demonstrated by the dataset.
Interview question
Which statement best describes how supervised fine-tuning updates a pre-trained language model?
- a.It trains on raw internet text using the same objective as pre-training but with a larger learning rate.
- b.It updates existing weights by predicting the next token only on curated assistant completions, masking the prompt.Correct
- c.It switches the training objective from causal language modeling to reward maximization.
- d.It freezes all base parameters and automatically adds new trainable layers to the model architecture.
Why? this is the answer
Supervised fine-tuning updates existing weights by continuing next-token prediction on curated prompt-completion data, computing loss only on assistant tokens while masking prompts. The most tempting distractor is wrong because SFT deliberately uses curated demonstrations and a small learning rate, not raw internet text like pre-training.
Just read this? Test yourself on what you have been reading.
Read the original → huggingface.co
- #llms
- #fine-tuning
- #sft
- #generative-ai
- #transformers
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on llms — each one lists the topics its interview covers.
See open roles