Frame Differencing: The Simplest Way to See Motion

WHY IT EXISTS To detect motion in a video stream in the simplest, most computationally inexpensive way possible. Before complex machine learning models, systems needed a fast and cheap method to answer the basic question: 'Did anything in the scene change between this moment and the last?'.

THE MENTAL MODEL Think of a 'spot the difference' picture puzzle. You have two nearly identical images, and your brain quickly finds the few elements that have changed. Frame differencing does the same thing computationally: it subtracts one frame from the next, and anything that isn't black in the resulting image is a 'difference'—an area of motion.

HOW IT WORKS The process involves a few simple steps. First, take two consecutive frames from a video, Frame A and Frame B. Second, for every single pixel, you calculate the absolute difference in intensity values between the two frames. This creates a new 'difference image' where pixels are bright if there was a change and black if there was none. Finally, you apply a threshold. Any pixel difference above this threshold is considered motion, filtering out minor camera noise and creating a binary mask that isolates the moving parts of the scene.

WHEN TO USE IT Use frame differencing when you have a static camera, a controlled environment, and can tolerate some errors. It's ideal for simple motion triggers, like in a basic security camera monitoring an empty room or a wildlife camera waiting for an animal to appear. It's also effective as a pre-processing step to identify 'regions of interest' that can be passed to more computationally expensive algorithms for further analysis.

WHEN NOT TO USE IT Avoid frame differencing in dynamic environments or when accuracy is critical. It performs poorly with camera jitter, as the entire scene will be registered as motion. It is also easily fooled by changes in lighting, such as clouds passing overhead or lights turning on and off, which create massive false positives. It also struggles to detect very slow-moving objects and can't identify a whole object, only the pixels that changed position between frames.

ONE CANONICAL EXAMPLE A simple security system monitors an empty hallway. Frame 100 shows the empty hall. Frame 101 shows a person who has just stepped into the frame. Subtracting the pixel values of Frame 100 from Frame 101 results in an image that is almost entirely black, except for a bright silhouette where the person is. The system counts the number of bright pixels; if it exceeds a set threshold, it triggers an alarm.