Disparity and depth in stereo vision

WHAT THIS TESTS The interviewer wants the precise geometric relationship that lets two cameras recover depth, and the often-confused inverse nature of it.

A GOOD ANSWER COVERS In a calibrated, rectified stereo pair the two cameras are horizontally offset by a known baseline and their image planes are aligned, so a single scene point projects to the same row in both images but at different horizontal positions. Disparity is the difference in horizontal pixel coordinates between the left-image and right-image projections of that point. The key relation is that depth, the distance from the camera to the point, is inversely proportional to disparity: depth equals the focal length times the baseline, divided by the disparity. Therefore nearby points shift a lot between the two views and have large disparity, while distant points shift little and have small disparity.

COMMON WRONG ANSWERS Claiming depth increases with disparity rather than being inversely related. Forgetting that the formula needs both the baseline and the focal length. Ignoring that rectification makes the search horizontal so disparity is purely horizontal.

LIKELY FOLLOW-UPS What happens to depth precision for far objects. Why does a wider baseline improve far-range accuracy. How is the disparity map computed by matching.

ONE CONCRETE EXAMPLE Consider a stereo camera with a fixed baseline whose two lenses look at a scene with a coffee mug close by and a wall far behind. The mug's edge appears, say, forty pixels to the left in the right image compared with the left image, a large disparity, while a point on the distant wall shifts only three pixels, a small disparity. Plugging into depth equals focal length times baseline over disparity, the large disparity yields a small depth, confirming the mug is near, and the tiny disparity yields a large depth, confirming the wall is far. A point at infinity would have zero disparity. This also explains why stereo depth is imprecise at long range: when disparity is only a few pixels, a one-pixel matching error changes the estimated depth substantially, which is why a wider baseline, increasing disparity for the same point, improves far-range accuracy. Computing disparity at every pixel produces a disparity map, which converts directly into a depth map.