Detector head losses: regression versus classification

WHAT THIS TESTS Whether you understand that detection jointly solves a classification and a regression problem and why each demands a different loss.

A GOOD ANSWER COVERS A modern detector head produces, per anchor or location, both a class prediction and a set of bounding box coordinates, so it is a multi-task output. The classification branch outputs scores over the category set and is trained with cross-entropy, or focal loss in dense detectors, because labels are discrete and cross-entropy directly optimizes the predicted probability of the correct class with well-behaved gradients. The box regression branch outputs continuous offsets, often relative to anchor boxes, and is trained with a regression loss. Smooth L1, also called Huber-style, behaves quadratically for small errors, giving stable gradients near the target, but linearly for large errors, so large coordinate errors and outliers do not produce exploding gradients the way pure L2 would. Modern detectors increasingly use IoU-family losses like GIoU or DIoU that optimize box overlap directly rather than coordinate differences. The two losses are combined as a weighted sum, and the classification loss is computed for all samples while regression is computed only for positive, object-containing samples.

COMMON WRONG ANSWERS Applying cross-entropy to continuous coordinates or L1 to class logits. Forgetting that regression loss applies only to positive anchors. Saying L2 is fine; its sensitivity to outliers is exactly why Smooth L1 exists. Ignoring the loss-weighting balance between the two tasks.

LIKELY FOLLOW-UPS Why is Smooth L1 preferred over pure L2 for boxes. Why compute regression loss only on positives. What advantage do IoU losses have over coordinate regression. How do you weight the two losses.

ONE CONCRETE EXAMPLE For a positive anchor matched to a car, the classification branch is penalized by cross-entropy for not assigning high probability to car, while the regression branch is penalized by Smooth L1 on the four offset values between the anchor and the true box. If one coordinate is off by many pixels, Smooth L1's linear regime keeps that gradient bounded, whereas L2 would square it into a destabilizing spike, illustrating why the robust loss is chosen for coordinates and cross-entropy for the categorical label.