Differential privacy vs utility in LLM fine-tuning

WHAT THIS TESTS This probes whether you understand differential privacy as a quantitative guarantee with an inherent accuracy cost, not a checkbox. It also tests whether you can reason about the mechanics of DP-SGD and the meaning of its hyperparameters.

A GOOD ANSWER COVERS DP-SGD modifies training in two ways: it clips each example's gradient to a maximum L2 norm so no single record dominates, then adds Gaussian noise calibrated to that clip bound before the optimizer step. Epsilon quantifies the worst-case privacy loss; smaller epsilon means a stronger guarantee but more noise and lower utility. Delta is the small probability the epsilon bound is violated, typically set well below one over the dataset size. A privacy accountant composes the per-step cost into a total budget across all epochs. To manage the trade-off, use large batch sizes so noise averages out, tune the clipping norm, limit epochs, and consider parameter-efficient methods like LoRA so fewer parameters absorb noise. For LLMs trained on user data, group examples by user to provide user-level rather than record-level privacy.

COMMON WRONG ANSWERS Saying DP is free; reversing epsilon so larger values are described as more private; forgetting gradient clipping and only mentioning noise; ignoring that the budget accumulates across steps; assuming a single epsilon value is universally safe.

LIKELY FOLLOW-UPS How does the accountant compose budget? What epsilon is reasonable for user data? How does batch size interact with noise? Why does LoRA help under DP?

ONE CONCRETE EXAMPLE Fine-tuning on customer support transcripts, you set a clipping norm, choose a target epsilon around a single-digit value with delta near one over ten million, and use a large physical-or-virtual batch. The accountant reports the spent budget after each epoch; you stop training once it nears the target, accepting a few points of accuracy loss in exchange for a provable bound on how much any one user's data influenced the model.