How does MAML's inner and outer loop work?

WHAT THIS TESTS The interviewer wants to know if you grasp meta-learning as bi-level optimization rather than as a fancier form of pretraining. MAML is model-agnostic, so it applies to any differentiable model trained with gradient descent.

A GOOD ANSWER COVERS MAML learns a parameter initialization theta that is good for fast adaptation. In the inner loop, for each sampled task you take one or a few gradient steps on that task's support set to produce adapted parameters theta-prime. In the outer loop you evaluate the loss of theta-prime on that task's query set and update the original theta to minimize the sum of these post-adaptation losses across the task batch. Because theta-prime is itself a function of theta, the outer gradient passes through the inner update, producing second-order derivatives. The objective is explicitly to find an init from which a small number of steps yields strong performance on new tasks.

COMMON WRONG ANSWERS Describing MAML as standard transfer learning or as training one network to solve every task at once. Forgetting that adaptation happens at test time on new tasks. Ignoring the second-order term, or not knowing first-order approximations like FOMAML and Reptile exist to cut that cost.

LIKELY FOLLOW-UPS Why is it expensive (Hessian-vector products through the inner steps), how FOMAML approximates it, how it differs from metric-based methods like Prototypical Networks, and how task and support-set sampling is structured for few-shot episodes.

ONE CONCRETE EXAMPLE In 5-way 1-shot image classification, each task is a fresh set of five classes. The inner loop fine-tunes on one image per class; the outer loop measures accuracy on held-out query images and nudges the init so that single shot adaptation generalizes. After meta-training, a brand-new five-class task can be learned from just one example per class with a handful of gradient steps.