SIFT versus SURF versus ORB

WHAT THIS TESTS The interviewer wants you to weigh accuracy, speed, memory, and licensing across three classic local-feature methods and justify a context-driven choice.

A GOOD ANSWER COVERS SIFT detects scale-space extrema and produces a 128-dimensional floating-point descriptor; it has excellent scale and rotation invariance and strong matching accuracy, but it is computationally heavy and the descriptor is large to store and compare. SURF was designed to approximate SIFT faster by using integral images and box filters; it is quicker than SIFT with similar but slightly lower robustness. ORB combines the FAST corner detector with the BRIEF binary descriptor, adding orientation for rotation invariance and a pyramid for partial scale handling. ORB descriptors are short binary strings compared with cheap Hamming distance, so it is dramatically faster and lighter, and it is unencumbered by patents.

COMMON WRONG ANSWERS Claiming ORB matches or exceeds SIFT accuracy generally; ORB trades some robustness, especially to large scale and viewpoint change, for speed. Ignoring the float-versus-binary descriptor distinction and the matching metric it implies. Forgetting the historical patent situation that pushed many projects to ORB.

LIKELY FOLLOW-UPS Why can binary descriptors be matched faster. How does ORB achieve rotation invariance with BRIEF. How does each degrade under severe viewpoint changes.

ONE CONCRETE EXAMPLE Suppose you are building a real-time augmented-reality feature on a mobile phone that must detect and track features at thirty frames per second within a tight CPU and battery budget. SIFT would likely be too slow and memory-hungry to sustain the frame rate on-device, and its license history complicated commercial use. ORB fits: FAST keypoints are cheap, the binary descriptor is tiny and compared with a few Hamming-distance operations, and the whole pipeline runs comfortably in real time on a phone CPU, with adequate rotation and moderate scale invariance for the task. You would accept slightly lower robustness to extreme zoom or oblique viewpoints in exchange for the speed and footprint. Conversely, for offline panorama stitching or high-precision matching where compute is not constrained, you would prefer SIFT for its superior accuracy and invariance.