How do you speed up a slow detection model?

WHAT THIS TESTS The interviewer wants a toolbox of optimization levers and, more importantly, honest trade-offs for each, showing engineering maturity.

A GOOD ANSWER COVERS Quantization reduces weights and activations from float32 to int8 or float16, shrinking the model and exploiting fast integer hardware; the trade-off is a usually small accuracy drop, mitigated by quantization-aware training. Pruning removes redundant weights or whole channels for a smaller, faster network; unstructured pruning needs sparse-aware hardware to pay off, while structured pruning gives reliable speedups at some accuracy cost. Knowledge distillation trains a compact student to mimic a large teacher, trading training effort for a faster model that retains much accuracy. Architecture changes, such as a MobileNet or EfficientNet backbone or a smaller YOLO variant, cut compute directly but cap peak accuracy. Reducing input resolution speeds everything but hurts small-object detection. Finally, runtime optimization with TensorRT, ONNX Runtime, operator fusion, and appropriate batching exploits the hardware; batching raises throughput but can increase per-frame latency.

COMMON WRONG ANSWERS Listing techniques with no trade-offs. Assuming quantization is free. Claiming unstructured pruning speeds up any hardware. Ignoring that the right choice depends on the target device.

LIKELY FOLLOW-UPS Quantization-aware versus post-training quantization, how to measure latency versus throughput, profiling to find bottlenecks, and accuracy-latency Pareto curves.

ONE CONCRETE EXAMPLE For a 30 fps requirement on an edge GPU, you might switch from a heavy two-stage detector to a one-stage YOLO with a MobileNet backbone, apply int8 quantization-aware training, and deploy through TensorRT with fused layers, accepting a couple of points of mAP loss to hit the frame-rate target.