Full Fine-Tuning: Updating Every Model Parameter
Full fine-tuning updates all weights of a pre-trained model on your new data, unlike methods that only change a small fraction. Use it to deeply embed new knowledge, but beware: it's costly and risks making the model forget its original general skills.
Full fine-tuning is the classic approach to specializing an LLM: you continue the training process on your custom dataset, updating every single one of the model's billions of parameters. This is for deep domain adaptation, like turning a general model into a legal or medical expert. The footgun is its high cost and the risk of "catastrophic forgetting," where the model unlearns its original world knowledge.
Read the original → arxiv.org
- #llm
- #fine-tuning
- #machine-learning
- #gen-ai
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.