N-way-K-shot: Classifying with Few Examples

Why it exists

Standard machine learning needs thousands of examples per class. But what if you need to identify a rare disease or a new product defect? You might only have a handful of images. N-way-K-shot provides a standardized way to measure a model's ability to learn in these data-scarce scenarios. It formalizes the problem of 'learning to learn.'

The mental model

Think of it like a flashcard quiz for an AI. You're not teaching it the entire subject of 'animals.' Instead, you show it one picture of a platypus (K=1) and one of an okapi (K=1), making it a 2-way, 1-shot problem. Then you give it a new picture and ask, 'Is this a platypus or an okapi?' The goal is to build a model that can pass these quizzes for any new set of classes it has never seen before.

How it works

The terminology breaks down like this: N-way refers to the number of distinct classes in a single classification task. K-shot refers to the number of labeled examples provided for each of those N classes. These examples form the 'support set.' The model is then given a new, unlabeled example (the 'query') and must assign it to one of the N classes. A common setup is a '5-way 1-shot' task: 5 classes, 1 example each.

When to use it

Use this framework to benchmark models on their few-shot learning capabilities. It's standard practice in meta-learning research. It's also a practical way to structure problems in domains with long-tail distributions of data, like e-commerce product categorization (many products have few images) or personalized robotics (a robot learning to recognize a user's specific coffee mug).

When not to use it

Do not use this as your primary training paradigm if you have abundant data. If you have thousands of labeled examples for each class, standard supervised classification will be more robust and perform better. N-way-K-shot is for evaluating learning agility, not for building a classifier on a large, static dataset. The biggest footgun is confusing this evaluation protocol with a training method.

One canonical example

A classic computer vision benchmark is a 5-way 1-shot task on the Omniglot dataset, which contains characters from many different alphabets. The model is shown 5 character classes it has never seen during training, with only one example of each. It must then classify a new, unseen instance of one of those 5 characters. This tests the model's ability to generalize the concept of 'same vs. different' rather than just memorizing specific character shapes.