Mean Average Precision in object detection

WHAT THIS TESTS Whether you understand a metric that combines precision, recall, ranking, and localization into one number, which is how detectors are compared.

A GOOD ANSWER COVERS For a single class, you sort all predicted boxes by confidence, and going down the list you match each to ground truth using an IoU threshold, marking true or false positives. As you lower the confidence threshold you trace out a precision-recall curve. Average Precision is the area under that curve, summarizing the precision-recall tradeoff for the class. Mean Average Precision is simply the AP averaged over all object classes. On COCO the headline metric, written mAP or AP, is stricter: it averages this over ten IoU thresholds stepping from zero point five to zero point nine five, so a detector is rewarded not just for finding objects but for tightly localizing them. COCO also reports AP at fixed thresholds and broken down by object size.

COMMON WRONG ANSWERS Calling mAP plain accuracy; it captures the full precision-recall tradeoff, not a single operating point. Forgetting it averages over classes. Ignoring COCO's averaging over IoU thresholds, which is what makes it demanding. Confusing AP at IoU zero point five with COCO's primary metric.

LIKELY FOLLOW-UPS Why use area under the PR curve rather than a single threshold. How does COCO's multi-IoU averaging differ from Pascal VOC. What is mAP at fifty versus mAP at seventy-five. How does the size breakdown help diagnosis.

ONE CONCRETE EXAMPLE For the class dog, ranking detections and sweeping confidence yields a PR curve whose area is, say, zero point seventy. Repeating for all eighty COCO classes and averaging gives mAP at IoU zero point five. To get COCO's primary number you recompute that mean at IoU thresholds zero point five, zero point five five, up to zero point nine five and average the ten results, so a model with loose boxes scores well at zero point five but loses points at zero point nine five, dragging down its overall AP.