Feature Maps: What a Neural Network 'Sees'

WHY IT EXISTS Neural networks need a way to process spatial information in data like images. Instead of looking at every pixel individually, they need to recognize patterns (lines, textures, shapes) regardless of where they appear. Feature maps are the mechanism for representing the presence and location of these learned patterns.

THE MENTAL MODEL Think of a feature map as the result of looking at an image through a special lens. One lens might only show you vertical lines. Another might only highlight bright red spots. A Convolutional Neural Network (CNN) learns to create thousands of these "lenses" (called filters) to deconstruct an image into its component patterns. Each feature map is the output from one of these lenses, showing "activations" where its specific pattern was found.

HOW IT WORKS A CNN applies a small filter (a small matrix of numbers, e.g., 3x3) across an input image. This filter is designed to detect a specific feature. At each position, the filter's values are multiplied with the underlying pixel values and summed up. A high sum (a high "activation") means the feature is strongly present at that location. The grid of all these activation values forms the feature map. Early layers learn simple filters (edges, corners), and their output feature maps become the input for the next layer, which learns to combine them into more complex features.

WHEN TO USE IT Feature maps are the core component of Convolutional Neural Networks (CNNs). They are fundamental to any task involving learning from spatial data, most commonly in computer vision for tasks like image classification, object detection, and segmentation. They also appear in models processing audio spectrograms or even some types of text data treated as a 1D "image".

WHEN NOT TO USE IT Feature maps are inherently tied to the convolutional architecture. They are less relevant for models that don't process data with strong spatial locality, like a simple feedforward network analyzing tabular data, or for architectures like Transformers that use different mechanisms (self-attention) to relate different parts of the input.

ONE CANONICAL EXAMPLE To detect a cat, a CNN's first layer might have filters that produce feature maps for horizontal lines, vertical lines, and simple curves. A subsequent layer takes these maps as input and has filters that learn to activate when a certain combination of curves and lines appears together, producing a feature map for "cat ear". Another map might activate for "whiskers". Deeper layers combine these to create a final "cat" feature representation.