Sliding Windows: Scanning Images for Objects

Why it exists

Image classification can tell you if an object is in an image, but not where. The sliding window technique was developed to solve this localization problem. It provides a systematic way to search an entire image to find the precise coordinates of an object.

The mental model

Think of searching for a specific person in a large crowd photo. You might take a small, rectangular cutout and scan it across the photo from left-to-right and top-to-bottom. At each spot, you check if the cutout perfectly frames the person you're looking for. A sliding window does this digitally: it's a fixed-size box that methodically scans an image, passing each patch to a classifier.

How it works

The algorithm defines a window of a fixed width and height (the windowSize). This window slides across the image, but to save computation, it doesn't move one pixel at a time. Instead, it jumps by a stepSize (e.g., 4 to 8 pixels) in both the x and y directions. At every position, the image patch inside the window is extracted and fed to a classifier. The classifier then decides if that patch contains the object of interest. This process repeats until the entire image has been scanned.

When to use it

Sliding windows are a foundational technique in classic object detection, especially when you need to find the exact location of objects. It's most powerful when combined with an image pyramid. By running the same fixed-size window over scaled-down versions of the image, you can detect objects at various sizes, not just the one that matches the window dimensions.

When not to use it

Avoid this method in performance-critical applications. The brute-force scanning is computationally expensive and slow compared to modern, single-shot detectors (like YOLO). If your system needs real-time detection on video streams, a sliding window approach will almost certainly be a bottleneck. It is also less efficient for objects with many different aspect ratios.

One canonical example

A classic use case is face detection. A window of, say, 64x64 pixels slides across an image. At each step, the 64x64 patch is sent to a classifier trained to recognize faces. If the classifier returns a high score, the window's coordinates are marked as a potential face location. By running this process on an image pyramid, the system can find both large faces in the original image and smaller faces that become 64x64 in the scaled-down versions.