Union File Systems: Docker's Layered Magic

WHY IT EXISTS To avoid massive duplication and enable efficiency. Without a union filesystem, creating 100 containers from a 1GB image would require 100GB of disk space. With it, you use 1GB for the shared image plus a tiny amount for each container's unique changes, saving disk space and speeding up container creation.

THE MENTAL MODEL Imagine stacking transparent sheets of paper. Each sheet is a read-only filesystem layer from a Docker image. When you run a container, you place a new, blank, writable transparent sheet on top. You see a merged view of all sheets below, but any changes you make—any new drawings—are only made on that top writable sheet. The original layers remain pristine.

HOW IT WORKS A union filesystem, such as OverlayFS used by Docker, merges multiple directory trees (layers) into one. When a process requests a file, the driver searches from the top (writable) layer down through the read-only image layers until it finds the file. If you modify a file from a lower layer, a "copy-on-write" (CoW) operation is triggered. The file is copied from its read-only layer to the top writable layer, and the changes are applied there. Deleting a file from a base layer simply creates a "whiteout" file in the writable layer, which hides the original.

WHEN TO USE IT This is the default storage model for Docker containers and is fundamental to how images and containers function. It's excellent for distributing application code and its dependencies, as the layers are immutable and shareable. It enables fast container startup and efficient use of disk space and network bandwidth.

WHEN NOT TO USE IT Avoid it for any write-intensive workload. The copy-on-write mechanism adds performance overhead for every initial write to a file that exists in a lower layer. For data that needs to persist and be performant, such as database files, application logs, or user uploads, always use Docker volumes, which bypass the union filesystem entirely.

ONE CANONICAL EXAMPLE Docker's OverlayFS storage driver. When you run docker run -it ubuntu bash, OverlayFS takes the read-only layers of the ubuntu image and stacks a new writable layer on top for your container session. If you run touch /tmp/newfile, that file is created only in the top writable layer. The underlying ubuntu image remains untouched.