PTQ versus QAT for model quantization

WHAT THIS TESTS The interviewer checks practical knowledge of model optimization trade-offs between development cost and accuracy preservation.

A GOOD ANSWER COVERS PTQ: start from a fully trained FP32 model and convert weights and activations to INT8 after the fact. It uses a small representative calibration dataset to estimate the dynamic ranges needed to pick scale and zero-point parameters. It is fast, needs no labels or backprop, and is easy to deploy, but quantization error can noticeably degrade accuracy for sensitive layers or networks with large activation outliers. QAT: insert fake-quantization operations into the graph that simulate INT8 rounding during forward passes while keeping gradients flowing, then train or fine-tune. The model thus learns weights that tolerate quantization, usually recovering most accuracy. It costs extra training time and data and a more complex pipeline. Decision: try PTQ first because it is cheap; if the accuracy loss exceeds your budget, especially for compact or low-bit models, move to QAT.

COMMON WRONG ANSWERS Saying PTQ is always lossless, ignoring that some models degrade. Or claiming QAT requires no data or training, when it specifically needs a training loop. Or conflating dynamic-range PTQ with full integer quantization.

LIKELY FOLLOW-UPS What does calibration estimate and why does it need representative data. Why do activation outliers hurt PTQ. How does the straight-through estimator let gradients flow through fake-quant ops. How do per-channel scales help.

ONE CONCRETE EXAMPLE You PTQ a MobileNet for a phone using a few hundred representative images for calibration, but accuracy drops several points, too much for the product, largely because compact models have little redundancy and some layers have wide activation outliers. You switch to QAT, inserting fake-quantization ops and fine-tuning for a few epochs so the weights learn to tolerate INT8 rounding, with the straight-through estimator letting gradients flow. Accuracy recovers to within a fraction of a point of FP32 while you keep the INT8 speedup and the smaller model size. Per-channel weight scales further reduce the loss compared with a single per-tensor scale.