Stereo rectification math and its artifacts

WHAT THIS TESTS Whether you grasp epipolar geometry beyond the textbook diagram and understand that rectification is a warp with real costs, not a free simplification.

A GOOD ANSWER COVERS Given a calibrated stereo pair, the epipolar constraint forces a point's match in the other image to lie on its epipolar line. Rectification computes two homographies that virtually rotate both image planes until they are coplanar and parallel to the baseline. This sends both epipoles to infinity in the horizontal direction, so all epipolar lines become parallel horizontal scanlines and corresponding points share the same row. Matching then reduces to a one dimensional search along a row, and disparity directly encodes depth. The homographies derive from the relative rotation and translation between cameras; Bouguet's method splits the rotation evenly to minimize distortion.

COMMON WRONG ANSWERS Saying rectification removes radial lens distortion; that is a separate undistortion step done first. Claiming the result is always lossless. Forgetting that rectification needs known relative pose or a fundamental matrix.

LIKELY FOLLOW-UPS Why send epipoles to infinity. How does disparity relate to depth and baseline. What happens to pixels with no valid mapping. How do you handle uncalibrated rectification.

ONE CONCRETE EXAMPLE Two cameras one meter apart, one rotated thirty degrees toward the other. The rectifying homographies must apply a large virtual rotation, so the warped images are heavily keystoned: one side is stretched, sampling fewer source pixels per output pixel and creating blur and aliasing, while wide black borders appear where the field of view no longer overlaps. The shared overlap shrinks and matching quality degrades at the stretched edges, illustrating why narrow baselines and small rotations rectify cleanly while wide ones do not.