Translation equivariance versus invariance in CNNs

WHAT THIS TESTS Whether you can be precise about a property that is widely stated loosely. The distinction between equivariance and invariance reveals real depth.

A GOOD ANSWER COVERS Translation equivariance means that if you shift the input, the output transforms in the same predictable way: shift the image, and the feature map shifts identically. Convolution with weight sharing is equivariant by construction, because the same filter is applied everywhere. Translation invariance means the output does not change at all when the input shifts, which is what a classifier wants for the final label. Convolution alone does not provide invariance; equivariance only becomes approximate invariance after operations that discard spatial location, namely pooling and especially the final global pooling or flattening before the classifier. So the network is equivariant in its convolutional body and invariant in its prediction only as a consequence of aggregation. Crucially, this invariance is imperfect: strided convolutions and pooling sample on a grid, so a one-pixel shift can change which samples are kept, and boundary padding breaks symmetry, meaning real CNNs are only partially shift-invariant.

COMMON WRONG ANSWERS Saying CNNs are fully translation invariant by design. Using equivariance and invariance interchangeably. Crediting convolution itself with invariance rather than the pooling and aggregation stages. Ignoring aliasing from downsampling.

LIKELY FOLLOW-UPS Why does strided sampling break shift-invariance. How do anti-aliasing or blur-pooling restore it. How does data augmentation compensate. Is convolution equivariant to rotation or scale.

ONE CONCRETE EXAMPLE Slide an input image one pixel to the right. The convolutional feature maps slide one pixel too, demonstrating equivariance. After global average pooling, the pooled vector is nearly identical, so the class prediction is unchanged, demonstrating approximate invariance. But shift the image by a non-multiple of the total stride and the strided subsampling can land on different pixels, perceptibly changing the logits, which is why papers on making convolutional networks shift-invariant add low-pass filtering before downsampling.