Supervised Fine-Tuning (SFT): Teaching a Model to Chat

WHY IT EXISTS: Base LLMs are trained on vast internet text to predict the next word. This makes them knowledgeable, but not necessarily helpful or conversational. They might continue a user's question instead of answering it. SFT solves this by teaching the model the specific task of being a helpful, instruction-following assistant.

THE MENTAL MODEL: Think of a base model as a brilliant but eccentric professor who has read every book but has no social skills. SFT is the process of giving this professor a set of flashcards showing good examples of questions and answers. By studying these examples, the professor learns how to structure their knowledge into helpful, conversational replies.

HOW IT WORKS: SFT is a form of transfer learning. You start with a pre-trained base model and create a dataset of high-quality prompt-response pairs. These pairs are often formatted into a specific chat template, with distinct roles like "user" and "assistant". The model is then trained on this dataset, adjusting its weights to minimize the difference between its generated response and the ideal response. It's "supervised" because every training example has a known "correct" answer.

WHEN TO USE IT: Use SFT to adapt a base model for a specific task, like creating a customer support chatbot, a code generation assistant, or a general-purpose instruction-following model. It is the primary method for teaching a model to adhere to a specific conversational format, style, and tone.

WHEN NOT TO USE IT: SFT is not for teaching a model new, domain-specific knowledge; it primarily teaches style and format. The model mostly uses knowledge already in its pre-trained weights. To add new facts, you'd look to techniques like Retrieval-Augmented Generation (RAG) or continued pre-training on a domain-specific corpus.

ONE CANONICAL EXAMPLE: A common SFT dataset might contain an entry like: [{"role": "user", "content": "Explain SFT in simple terms."}, {"role": "assistant", "content": "SFT is like showing a smart student examples of good test answers so they learn how to answer questions well, not just what the information is."}] The model learns to generate the assistant's content when given the user's content, following the specified role-based format.