PTQ versus QAT for INT8 quantization

WHAT THIS TESTS The interviewer wants a pragmatic decision framework, not just definitions. The signal is whether you start with the cheap option and escalate only when accuracy demands it.

A GOOD ANSWER COVERS Post-Training Quantization: take a fully trained FP16 model and convert weights, and usually activations, to INT8 after the fact. Activation ranges are estimated from a small representative calibration dataset run through the model. PTQ is fast, requires little data, and needs no gradient updates, so it is cheap and quick, but because the model never adapted to quantization noise it can suffer larger accuracy degradation, especially with outlier-heavy activations. Quantization-Aware Training: insert fake-quantization operations into the forward pass during training or fine-tuning so the model sees rounding effects and learns weights robust to them, using a straight-through estimator for gradients. QAT typically recovers most or all of the lost accuracy but demands training infrastructure, labeled or representative data, and significant compute and time. Decision: default to PTQ because it is cheap and often sufficient for INT8; measure the accuracy hit on your evaluation set; if the drop is within tolerance, ship it; if PTQ degrades quality unacceptably, the model is small or accuracy-critical, or you target very low bit-widths, invest in QAT.

COMMON WRONG ANSWERS Claiming QAT skips training, it is defined by training with simulated quantization. Saying PTQ always matches QAT accuracy. Ignoring calibration data for PTQ. Recommending QAT first regardless of cost.

LIKELY FOLLOW-UPS What does the calibration set do in PTQ, estimate activation ranges. Why is QAT more accurate, the model adapts to quantization noise. How do outliers motivate advanced PTQ like SmoothQuant or GPTQ.

ONE CONCRETE EXAMPLE INT8 PTQ on a large model loses under one point on your benchmark, well within budget, so you ship PTQ. On a tiny on-device model the same PTQ loses several points, so you run QAT to recover accuracy before deployment.