MLP-Mixer: Vision Without Convolutions or Attention

WHY IT EXISTS MLP-Mixer was created to challenge a core assumption in computer vision: that specialized operations like convolutions (in CNNs) or self-attention (in Transformers) are essential for high performance. The goal was to demonstrate that a much simpler, more general component—the Multi-Layer Perceptron (MLP)—could be sufficient, potentially opening up new avenues for architectural design.

THE MENTAL MODEL Think of MLP-Mixer as a two-step communication process for image patches. First, each patch 'talks to itself' to refine its own features (channel-mixing). Then, all the patches 'talk to each other' to share spatial information and understand their context within the whole image (token-mixing). This cycle of internal and external communication repeats until the model understands the image content.

HOW IT WORKS The architecture first divides an input image into a grid of non-overlapping patches. Each patch is linearly projected into a feature vector. These vectors are then fed through a series of identical 'Mixer' layers. Each Mixer layer contains two distinct MLP blocks. The first, the 'token-mixing' MLP, operates across patches, allowing spatial information to be shared. The second, the 'channel-mixing' MLP, operates on each patch independently, mixing the features within that patch. By stacking these layers, the model progressively integrates local and global information to perform classification.

WHEN TO USE IT Use MLP-Mixer when you have access to massive datasets (in the hundreds of millions or billions of images) and the computational budget for large-scale pre-training. In this scenario, its performance can be competitive with state-of-the-art CNNs and Transformers. Its architectural simplicity can also be an advantage for research and analysis.

WHEN NOT TO USE IT Avoid MLP-Mixer for projects with small or medium-sized datasets. Unlike CNNs, which have a strong 'inductive bias' for vision that helps them learn efficiently, MLP-Mixer has very weak priors. It learns visual patterns from scratch, which requires seeing an enormous amount of data to generalize effectively. On smaller datasets, it will almost always underperform a standard CNN.

ONE CANONICAL EXAMPLE A typical application involves pre-training an MLP-Mixer model on a huge dataset like JFT-300M or ImageNet-21k. This model, now possessing a general understanding of visual features, is then fine-tuned on a smaller, specific task, such as classifying different types of aircraft. The pre-training is not optional; it is the essential step that makes the architecture viable.