Incremental Structure from Motion pipeline
3D reconstruction workflow.
detect and match features, estimate two-view geometry, triangulate, then incrementally add images with PnP and refine via bundle adjustment.
What's really being asked
The interviewer wants a clear, ordered understanding of how unordered 2D photos become camera poses and a 3D point cloud, including the role of optimization.
The full answer
The input is a collection of overlapping images of a static scene. The pipeline begins with feature extraction and matching: detect keypoints such as SIFT in every image and match descriptors between image pairs, filtering with the ratio test and geometric verification using the fundamental or essential matrix and RANSAC. Then choose a good initial image pair, estimate their relative pose from the essential matrix, and triangulate the matched points to seed an initial sparse 3D structure. The incremental step repeats: select the next image with enough correspondences to existing 3D points, estimate its pose by solving the Perspective-n-Point problem, triangulate newly visible points, and periodically run bundle adjustment, a nonlinear least-squares optimization that jointly refines all camera poses and 3D point positions to minimize reprojection error. The outputs are the camera poses, intrinsics if also solved, and a sparse 3D point cloud.
The mistakes people make
Omitting bundle adjustment, which is what keeps drift in check. Confusing the sparse SfM point cloud with the dense reconstruction produced later by multi-view stereo. Forgetting geometric verification of matches.
What usually comes next
Why is bundle adjustment necessary and what does it minimize. How does PnP register a new camera. How does incremental SfM differ from global SfM.
A concrete example
Suppose you have a hundred photos of a statue taken while walking around it. The pipeline detects and matches features across overlapping shots, picks two images with a wide-enough baseline and many shared points as the seed, recovers their relative rotation and translation from the essential matrix, and triangulates the shared features into an initial cloud of 3D points on the statue. It then adds a third image by matching its features to the existing 3D points and solving PnP for that camera's pose, triangulates the points it newly sees, and runs bundle adjustment to nudge every camera and point so that all 3D points reproject as closely as possible to their detected image locations. Repeating this across all hundred images yields the pose of every camera and a sparse 3D point cloud outlining the statue. A subsequent multi-view stereo stage can densify this into a full surface, but classical SfM itself outputs the poses and the sparse structure.
Interview question
In an incremental SfM pipeline, what is the primary role of bundle adjustment?
- a.To densify the sparse point cloud into a full surface mesh
- b.To choose the initial image pair for reconstruction
- c.To detect and match feature descriptors across images
- d.To jointly refine all camera poses and 3D points by minimizing reprojection errorCorrect
Why? this is the answer
Bundle adjustment is the nonlinear optimization that simultaneously adjusts poses and points to reduce reprojection error and control drift. Densification is a later multi-view stereo step, and matching and seed selection happen before it.
Just read this? Test yourself on what you have been reading.
Read the original → theia-sfm.org
- #structure-from-motion
- #bundle-adjustment
- #triangulation
- #pnp
- #3d-reconstruction
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles