Fundamental matrix versus essential matrix

WHAT THIS TESTS The interviewer wants you to separate the calibrated and uncalibrated cases of two-view geometry and know what each matrix gives you.

A GOOD ANSWER COVERS Both matrices algebraically express the epipolar constraint, that a point in one image and its match in the other satisfy a bilinear relation, which is what confines a match to its epipolar line. The fundamental matrix relates corresponding points expressed in pixel coordinates. Because it absorbs the unknown camera intrinsics, you can estimate it directly from a set of point correspondences alone, with no calibration, using methods like the eight-point algorithm. The essential matrix relates corresponding points expressed in normalized, calibrated coordinates, so computing it requires knowing the camera intrinsic matrices. Its payoff is that it factors directly into the relative rotation and translation between the two cameras, recovering pose up to a scale.

COMMON WRONG ANSWERS Reversing which matrix needs intrinsics. Claiming the fundamental matrix gives relative pose directly, when only the essential matrix decomposes cleanly into rotation and translation. Forgetting that translation from the essential matrix is recovered only up to scale.

LIKELY FOLLOW-UPS How are the two matrices related through the intrinsic matrices. How do you decompose the essential matrix into rotation and translation. Why is the eight-point algorithm normalized.

ONE CONCRETE EXAMPLE Suppose you have two photos of a building from unknown cameras and a set of matched feature points. With no calibration you can estimate the fundamental matrix from those matches, then for any point draw its epipolar line in the other image, which is enough to guide matching and to verify geometric consistency. Now suppose you also know the cameras' intrinsics, focal lengths and principal points. You convert the matched pixels to normalized coordinates and compute the essential matrix, or equivalently obtain it from the fundamental matrix by multiplying with the intrinsic matrices. Decomposing the essential matrix yields the rotation and the translation direction between the two camera positions, so you can reconstruct the relative camera motion and triangulate 3D points up to an overall scale ambiguity. This is exactly the step structure-from-motion uses to bootstrap a reconstruction: fundamental for uncalibrated relating, essential for calibrated pose recovery.