Contrastive learning vs masked image modeling

WHAT THIS TESTS The interviewer wants a precise contrast of two dominant self-supervised paradigms, including their pretext tasks and the practical trade-offs that guide which to use.

A GOOD ANSWER COVERS Contrastive (MoCo): the pretext task is instance discrimination. Two augmented views of the same image form a positive pair that should be close in embedding space, while views of other images are negatives pushed apart; MoCo uses a momentum encoder and a queue to provide many negatives. It depends strongly on carefully tuned augmentations and tends to produce features that perform well under linear probing. Masked image modeling (MAE): the pretext task is reconstruction. A high fraction of image patches, often 75 percent, are masked, and an asymmetric encoder-decoder reconstructs the missing pixels. It needs minimal augmentation, scales efficiently because the encoder sees only visible patches, suits Vision Transformers naturally, and frequently yields better end-to-end fine-tuning accuracy.

COMMON WRONG ANSWERS Saying both just learn good features without naming the distinct pretext tasks. Or claiming MAE needs heavy augmentation like contrastive methods, which misses a key difference. Or asserting one is always superior rather than noting linear-probe versus fine-tuning trade-offs.

LIKELY FOLLOW-UPS Why does contrastive learning need many negatives and how does MoCo supply them. Why does MAE use such a high mask ratio. Why is MAE efficient on ViTs. Which gives better linear-probe versus fine-tuning results and why.

ONE CONCRETE EXAMPLE To pretrain a ViT on unlabeled images, MAE masks 75 percent of patches and reconstructs the missing pixels, training efficiently because the heavy encoder processes only the visible 25 percent of patches and a lightweight decoder handles the rest, then it fine-tunes strongly on ImageNet. A MoCo alternative would instead build positive pairs from aggressive crops and color jitter of the same image, push apart a large queue of negatives via a momentum encoder, and often give better frozen-feature linear probes even if MAE wins on full fine-tuning. The right choice depends on whether you will fine-tune or use frozen features.