Harris corner detector and corner stability

WHAT THIS TESTS The interviewer checks whether you understand what makes a feature distinctive and trackable, and the math behind the classic corner response.

A GOOD ANSWER COVERS Harris considers a small window and asks how much the summed intensity would change if you shifted the window slightly in any direction. This is captured by the structure tensor, a 2 by 2 matrix built from the products of the horizontal and vertical image gradients summed over the window. The two eigenvalues of this matrix describe how intensity varies along two principal directions. A flat region has both eigenvalues small, an edge has one large and one small, and a corner has both large. Harris uses a corner response score combining the determinant and trace to flag points where both eigenvalues are large, avoiding explicit eigenvalue computation.

COMMON WRONG ANSWERS Saying it detects edges. Claiming it works on color directly rather than intensity gradients. Forgetting that corners are stable precisely because they constrain position in two dimensions.

LIKELY FOLLOW-UPS What is the aperture problem and why do edges suffer from it. Is Harris scale-invariant. How does the response threshold and non-maximum suppression pick final points.

ONE CONCRETE EXAMPLE Imagine tracking the corner of a window frame versus a point along a straight horizontal wire. At the corner, any small movement of the tracking window, left, right, up, or down, produces a large change in the windowed intensity, so the matcher can pin the location precisely in both x and y. Along the wire, you can slide the window left or right and the intensity barely changes because the structure looks the same; you only know your position perpendicular to the wire, not along it. This ambiguity is the aperture problem, and it is exactly why edges drift during tracking while corners stay locked. The Harris structure tensor encodes this: at the corner both eigenvalues are large, giving a high response, whereas along the wire one eigenvalue is near zero, so Harris correctly rejects it. That two-directional constraint is what makes corners reliable interest points for matching and tracking.