Image classification versus object detection

WHAT THIS TESTS Whether you can crisply separate two tasks that beginners blur together, which underpins understanding detection architectures.

A GOOD ANSWER COVERS Image classification takes an image and assigns it one label from a fixed set, typically outputting a probability distribution over all classes via a softmax, from which the top class is read. It assumes a single dominant subject and says nothing about location. Object detection is harder: it must find every object of interest, draw a bounding box around each, assign each box a class label, and attach a confidence score. So its output is a variable-length list of detections, each a tuple of box coordinates, class, and score, and it handles many objects of possibly different classes in one image. The two added requirements over classification are localization, knowing where, and multiplicity, handling several instances at once.

COMMON WRONG ANSWERS Saying detection is just classification with more classes. Omitting that detection localizes with boxes. Forgetting detection outputs a variable number of objects. Confusing detection with segmentation, which labels pixels rather than boxes.

LIKELY FOLLOW-UPS How do you evaluate detection versus classification. What is a bounding box format. How does detection handle a variable number of outputs. Where does segmentation fit relative to these two.

ONE CONCRETE EXAMPLE Given a street photo, a classifier might output the single label street scene or its most prominent class such as car. A detector instead outputs several boxes: one around each of three cars, one around a pedestrian, and one around a traffic light, each box tagged with its class and a confidence like zero point nine. The classifier gives one answer for the whole image; the detector gives a labeled, localized list of every object it finds.