Leveraging unlabeled data with limited labels

WHAT THIS TESTS: Whether you know how to turn a large unlabeled corpus into a performance gain when labels are scarce.

A GOOD ANSWER COVERS: Training a supervised model on only a thousand labels overfits and wastes the million unlabeled images, so the goal is to extract knowledge from the unlabeled set. The strongest general approach is self-supervised pretraining: define a pretext task that needs no labels, such as contrastive learning where augmented views of the same image are pulled together and different images pushed apart, or masked-image modeling, and train an encoder on all million images to learn rich representations. Then fine-tune that encoder on the thousand labeled examples, optionally freezing most of it and training a small head first. Complementary semi-supervised techniques can be layered on: pseudo-labeling uses the current model to label confident unlabeled images and adds them to training; consistency regularization enforces that the model gives the same prediction for two augmentations of an unlabeled image; modern methods combine confident pseudo-labels with strong augmentation. Either way the unlabeled data shapes the feature space, so the few labels only need to fit a decision boundary rather than learn features from scratch.

COMMON WRONG ANSWERS: Ignoring the unlabeled data and training purely on the thousand labels; pseudo-labeling without confidence thresholds, which amplifies the model's own errors; assuming you must hand-label more data; conflating semi-supervised learning with simple data augmentation of the labeled set alone.

LIKELY FOLLOW-UPS: Why does self-supervised pretraining help when labels are scarce? How do you stop pseudo-labels from reinforcing mistakes via confirmation bias? What augmentations matter for contrastive learning? How would you evaluate the pretrained representation, for example with linear probing?

ONE CONCRETE EXAMPLE: Run a contrastive method like SimCLR over all one million images, learning an encoder with no labels. Then attach a classifier and fine-tune on the thousand labeled images, vastly outperforming a model trained on those thousand alone. To squeeze more, pseudo-label only unlabeled images the fine-tuned model classifies with high confidence and retrain, while using consistency regularization so predictions stay stable under cropping and color jitter.