How does Mask R-CNN do instance segmentation?

WHAT THIS TESTS Whether you understand how a detection framework is extended to produce per-instance pixel masks, and why small architectural choices like RoIAlign matter.

A GOOD ANSWER COVERS Mask R-CNN builds directly on Faster R-CNN. A convolutional backbone, often a ResNet with a Feature Pyramid Network, extracts features. A Region Proposal Network proposes candidate object regions. For each proposal, features are extracted and fed to two existing heads that predict the class and refine the bounding box. Mask R-CNN adds a third head, a small fully convolutional network, that outputs a binary mask for each RoI. Crucially, masks are predicted per class without competition between classes; the classification head selects which mask to use. RoIPool is replaced by RoIAlign, which uses bilinear interpolation instead of harsh quantization, preserving spatial alignment that pixel-accurate masks need.

COMMON WRONG ANSWERS Saying the mask branch uses a softmax across classes per pixel, which couples classification and segmentation unnecessarily. Forgetting RoIAlign and claiming RoIPool is fine, when its rounding introduces misalignment that degrades masks. Describing it as a single-stage detector or confusing it with semantic segmentation that ignores instances.

LIKELY FOLLOW-UPS Why does decoupling mask and class prediction help? How does RoIAlign differ numerically from RoIPool? How is the multi-task loss combining classification, box regression, and mask terms balanced? How does it handle overlapping instances?

ONE CONCRETE EXAMPLE For an image with two overlapping people, the RPN proposes regions around each. RoIAlign crops aligned features per region. The mask head outputs, say, a 28 by 28 mask per class for each RoI; the class head says person, so the person-channel mask is upsampled back to image coordinates, yielding two distinct instance masks even where the people overlap.