Kalman filter for bounding-box tracking

WHAT THIS TESTS Whether you understand the Kalman filter's predict-update cycle and can instantiate its matrices for a concrete tracking problem with noisy detections.

A GOOD ANSWER COVERS The Kalman filter maintains a Gaussian belief over the state: a mean state estimate and a covariance encoding uncertainty. Its core pieces are the state transition model that predicts how the state evolves, the process noise covariance capturing unmodeled dynamics, the measurement model mapping state to observation, and the measurement noise covariance capturing detector noise. Each frame runs two steps. Predict: propagate the state through the transition model and grow the covariance by the process noise. Update: compare the actual measurement to the predicted measurement, compute the Kalman gain from the relative uncertainties, and correct the state, shrinking the covariance. The gain automatically trusts the prediction more when measurements are noisy and vice versa.

STATE AND MEASUREMENT For a bounding box, a common state vector holds the box center x and y, width and height (or aspect and scale), and their time derivatives, the velocities, assuming constant velocity. The measurement vector is what the detector directly gives each frame, the box center, width, and height, without velocities. The measurement model selects the observed components from the state.

COMMON WRONG ANSWERS Swapping process and measurement noise. Putting only position in the state, so it cannot predict motion. Treating the gain as a fixed constant instead of computed from covariances.

LIKELY FOLLOW-UPS Why model velocity? What if motion is nonlinear, leading to the extended or unscented Kalman filter? How does SORT use this? How do you initialize the covariance for a new track?

ONE CONCRETE EXAMPLE Tracking a walking person, the state is center, size, and velocities. The constant-velocity model predicts the next center from current velocity. The noisy detection box arrives; the gain blends prediction and detection, smoothing jitter and bridging a one-frame missed detection by coasting on the predicted velocity.