Contrastive learning vs masked image modeling
self-supervised pretraining.
contrastive aligns augmented views via instance discrimination; MAE reconstructs masked patches; they differ in augmentation and fine-tuning.
conflating them or saying one always wins.
What's really being asked
The interviewer wants a precise contrast of two dominant self-supervised paradigms, including their pretext tasks and the practical trade-offs that guide which to use.
The full answer
Contrastive (MoCo): the pretext task is instance discrimination. Two augmented views of the same image form a positive pair that should be close in embedding space, while views of other images are negatives pushed apart; MoCo uses a momentum encoder and a queue to provide many negatives. It depends strongly on carefully tuned augmentations and tends to produce features that perform well under linear probing. Masked image modeling (MAE): the pretext task is reconstruction. A high fraction of image patches, often 75 percent, are masked, and an asymmetric encoder-decoder reconstructs the missing pixels. It needs minimal augmentation, scales efficiently because the encoder sees only visible patches, suits Vision Transformers naturally, and frequently yields better end-to-end fine-tuning accuracy.
The mistakes people make
Saying both just learn good features without naming the distinct pretext tasks. Or claiming MAE needs heavy augmentation like contrastive methods, which misses a key difference. Or asserting one is always superior rather than noting linear-probe versus fine-tuning trade-offs.
What usually comes next
Why does contrastive learning need many negatives and how does MoCo supply them. Why does MAE use such a high mask ratio. Why is MAE efficient on ViTs. Which gives better linear-probe versus fine-tuning results and why.
A concrete example
To pretrain a ViT on unlabeled images, MAE masks 75 percent of patches and reconstructs the missing pixels, training efficiently because the heavy encoder processes only the visible 25 percent of patches and a lightweight decoder handles the rest, then it fine-tunes strongly on ImageNet. A MoCo alternative would instead build positive pairs from aggressive crops and color jitter of the same image, push apart a large queue of negatives via a momentum encoder, and often give better frozen-feature linear probes even if MAE wins on full fine-tuning. The right choice depends on whether you will fine-tune or use frozen features.
Interview question
Which statement best captures a fundamental difference between MoCo and MAE pretext tasks?
- a.MoCo discriminates augmented instances; MAE reconstructs masked patchesCorrect
- b.MoCo reconstructs pixels while MAE discriminates instances
- c.Both require heavy color and crop augmentations to work
- d.Both rely equally on a large negative queue
Why? this is the answer
MoCo uses instance discrimination with negatives and strong augmentation, while MAE is generative, reconstructing masked patches with little augmentation. The first option reverses them and the others ignore MAE's design.
Just read this? Test yourself on what you have been reading.
Read the original → arxiv.org
- #self-supervised
- #contrastive-learning
- #masked-autoencoder
- #pretraining
- #vit
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles