Homography: Mapping Flat Surfaces Between Images

Why it exists

Computer vision needs a way to relate two different images of the same scene. If that scene is flat, like a wall, a floor, or a piece of paper, a homography provides the exact mathematical transformation to map every point from one view to the other.

The mental model

A homography is a perspective transformation matrix. Imagine you take a photo of a rectangular poster from an angle, making it look like a trapezoid. The homography is the 3x3 matrix that can transform that trapezoidal shape in your photo back into a perfect rectangle, or vice-versa. It's a single operation that handles rotation, translation, scaling, and perspective skew all at once for a planar surface.

How it works

A homography is a 3x3 matrix that transforms 2D points represented in homogeneous coordinates (x, y, 1). To find this matrix, you need at least four pairs of corresponding points between the two images that are not collinear. For example, the four corners of a window in image A and the same four corners in image B. With these points, algorithms like Direct Linear Transform (DLT) can solve for the eight unknown values in the homography matrix. Once you have the matrix, you can use it to warp one entire image to align with the other.

When to use it

Use it for image registration, like stitching photos into a panorama by warping them to align perfectly. Use it for image rectification, like in a document scanner app that turns an angled photo of a page into a flat, top-down view. It is also fundamental to augmented reality for projecting virtual decals or objects onto a real-world floor or wall.

When not to use it

Do not use a single homography for scenes with significant depth or parallax. If you try to map a landscape with a foreground tree and a background mountain, the homography will fail because the points do not lie on a single plane. This results in ghosting or tearing artifacts. For such scenes, more complex methods like fundamental matrix estimation or 3D reconstruction are needed.

One canonical example

Creating a skybox in a video game by projecting a texture onto a cube. To make the texture on one face of the cube appear correct from the player's viewpoint, the game engine calculates a homography between the texture's flat plane and the polygon's plane on screen. This is also how virtual ads are inserted onto billboards or sports fields in live video broadcasts; the advertiser's image is warped by a homography to match the perspective of the surface in the video frame.