Harris Corner Detector: Finding Sharp Changes in Images

WHY IT EXISTS Computer vision systems need reliable "landmarks" in an image to perform tasks like tracking objects or stitching photos. Flat areas are useless, and edges are only distinct in one direction. Corners, however, provide a unique point that's stable in two dimensions, making them excellent anchor points.

THE MENTAL MODEL Imagine a small window sliding across an image. If the window is on a flat surface, its contents don't change as you slide it. If it's on an edge, the contents change when you slide it perpendicularly, but not along the edge. If the window is on a corner, its contents change drastically no matter which way you slide it. The Harris detector mathematically scores points based on this "drastic change in all directions" principle.

HOW IT WORKS For each pixel, the algorithm considers a small window around it. It calculates how much the image intensity changes (the gradient) in both the X and Y directions. It then analyzes these gradients to determine if there's a significant change in all directions. A large change in only one direction signifies an edge. Small changes in all directions signify a flat area. A large change in multiple directions signifies a corner. This was an improvement on the earlier Moravec detector, which only checked for changes at fixed 45-degree angles.

WHEN TO USE IT Use it as a fast and effective feature detector when you don't need your system to be robust to changes in image scale. It's a classic, reliable choice for preprocessing in applications like simple object tracking, stereo matching, or as a building block for more complex algorithms where the camera's distance to the subject is relatively constant.

WHEN NOT TO USE IT Avoid using it alone when your application must handle images at different sizes or zoom levels. A feature identified as a corner at one resolution might not be detected if you zoom in or out. For these cases, scale-invariant feature detectors like SIFT (Scale-Invariant Feature Transform) are more appropriate.

ONE CANONICAL EXAMPLE A panorama stitching application. To combine multiple photos, the software must find common points between them. It can run the Harris detector on each image to find a set of candidate corners. It then compares the regions around these corners to find matches, allowing it to calculate how to warp and align the images into a single seamless panorama.