When would you use LoRA vs full fine-tuning?

### What this tests This probes your understanding of the practical engineering and financial trade-offs in adapting large language models. It's not just about knowing what LoRA is, but *when* and *why* to apply it over the traditional, more expensive full fine-tuning approach. The interviewer is looking for a cost-benefit analysis rooted in system constraints.

### A good answer covers * **Core Trade-off:** Frame the decision primarily around cost (compute, memory, storage) vs. the required depth of model adaptation. LoRA is for efficiency; full fine-tuning is for maximum investment. * **Choose LoRA when:** * **Resource Constrained:** You have a limited GPU budget. LoRA can reduce the number of trainable parameters by 10,000x and GPU memory requirements by 3x for a GPT-3 175B model. * **Deploying Multiple Tasks:** You need many specialized models. Storing a small LoRA adapter (a few MB) for each task is far cheaper than storing a full 175B+ parameter model (hundreds of GB) for each. * **Fast Iteration is Key:** Training is significantly faster due to the small parameter count, enabling quicker experimentation and deployment cycles. * **Inference Latency is Critical:** LoRA adds no additional inference latency compared to the base model, a key advantage over other methods like adapters. * **Choose Full Fine-Tuning when:** * **Budget is Not a Constraint:** You have the resources (e.g., a large cluster of A100s/H100s) to retrain all 175B+ parameters and the associated time/cost is acceptable. * **Maximum Performance is Hypothesized:** You believe the target task requires updating all model weights for peak performance and are willing to pay the cost to test that hypothesis (though LoRA often performs on-par). * **A Single, Powerful Model is the Goal:** You are creating one monolithic, highly specialized model, not a suite of smaller ones.

### Common wrong answers * **Vague statements:** "LoRA is just a cheaper way to fine-tune." This lacks the critical "how" (injecting low-rank matrices) and "how much" (10,000x fewer params). * **Confusing with other methods:** Incorrectly stating that LoRA adds inference latency. This is a characteristic of classic adapter methods, which LoRA was designed to improve upon by merging weights. * **Ignoring the multi-task deployment benefit:** Missing the key operational advantage of swapping small LoRA weights for different tasks on top of a single, shared base model instance.

### Follow-up the interviewer might ask * "How do you choose the rank `r` for the LoRA matrices? What are the trade-offs?" * "You mentioned LoRA has no inference latency. How is that achieved architecturally?"

### One concrete example "For a 175B parameter model like GPT-3, a full fine-tune trains all 175B parameters. With LoRA, we freeze the 175B parameters and only train the injected low-rank matrices. This can reduce the trainable parameters by a factor of 10,000, meaning we might only train ~17.5 million parameters. This reduces the GPU memory requirement by 3x, making it feasible to train on a single multi-GPU server instead of a large, expensive cluster. For deployment, we can serve one base model and dynamically load different multi-megabyte LoRA adapters for different user requests, instead of needing separate 350GB model instances for each task."