Triangulation: Finding 3D Points from 2D Images

WHY IT EXISTS Cameras capture a 3D world onto a 2D plane, losing all depth information in the process. To understand a scene for robotics, 3D modeling, or measurement, we need a reliable way to reverse this and recover the lost third dimension from flat images.

THE MENTAL MODEL Imagine pointing at a distant object with both your index fingers, one representing each eye's line of sight. The point in space where your fingers would meet is the triangulated position. In computer vision, each camera provides a "ray" to the object; the 3D point is where these rays intersect.

HOW IT WORKS Given a point in 3D space, it projects to a specific pixel coordinate in each camera's 2D image. Triangulation reverses this process. If you know the 3D position, orientation, and lens properties of each camera (its "camera matrix"), you can trace a line from each camera's center through the corresponding pixel on its image plane out into the world. The 3D point is the intersection of these lines. In reality, noise means the lines won't perfectly intersect, so algorithms find the 3D point that minimizes the distance to all rays.

WHEN TO USE IT Use triangulation when you have multiple, calibrated camera views of the same scene and need to determine its 3D structure. This is fundamental for stereo vision systems in robots, 3D scanners, and creating 3D models from a set of photographs (a process called photogrammetry).

WHEN NOT TO USE IT Triangulation is impossible with only a single image, as you cannot recover depth from one viewpoint without other strong assumptions. It is also ineffective if you cannot reliably find and match the same feature point across multiple images, or if the camera parameters are unknown or inaccurate.

ONE CANONICAL EXAMPLE A self-driving car's stereo camera system. Two cameras, mounted a fixed distance apart, capture images simultaneously. The system identifies a feature, like the corner of a stop sign, in both images. By knowing the pixel coordinates of the corner in each image and the pre-calibrated camera setup, triangulation calculates the precise 3D position of that corner, determining its exact distance from the car.