Explain panoptic segmentation and Panoptic Quality

WHAT THIS TESTS Whether you grasp how panoptic segmentation unifies the stuff-versus-things split and whether you can decompose its metric rather than reciting it.

A GOOD ANSWER COVERS Panoptic segmentation assigns every pixel both a semantic label and, for things like cars or people, a unique instance id; stuff classes like sky or road receive only a class with no instances. This combines semantic segmentation, which ignores instances, and instance segmentation, which ignores stuff, into one coherent map. Panoptic Quality evaluates it by first matching predicted segments to ground truth: a match requires intersection-over-union strictly greater than 0.5, which guarantees a unique match per segment. PQ then factors cleanly into two interpretable parts. Segmentation Quality is the average IoU over all matched true-positive segments, measuring how tight the matched masks are. Recognition Quality is an F1 score over segments, computed as true positives divided by true positives plus half false positives plus half false negatives, measuring detection correctness. PQ equals SQ multiplied by RQ.

COMMON WRONG ANSWERS Treating PQ as just mIoU, which ignores instance counting and false detections. Forgetting the strict IoU above 0.5 matching rule that makes matches unique. Saying stuff classes get instance ids.

LIKELY FOLLOW-UPS Why does IoU above 0.5 guarantee at most one match? How does PQ penalize a missed instance versus a duplicate detection? How do panoptic architectures like Panoptic FPN or Mask2Former produce non-overlapping output?

ONE CONCRETE EXAMPLE A street scene with three cars, predicting two with tight masks and one missed, plus one false car: RQ drops from both the false negative and false positive, SQ reflects the two good IoUs, and PQ multiplies them, lower than mIoU would suggest because it counts the recognition mistakes.