Kubernetes Jobs: For Tasks That Need to Finish

Why it exists

Not all work is a long-running service. Some tasks, like processing a file, running a database migration, or creating a backup, need to run once and then stop. Kubernetes needed a way to manage these finite tasks, ensuring they complete successfully without needing to be manually supervised.

The mental model

A Job is a supervisor for a temporary task. You give it a Pod template and say, "Run this until it exits with a success code." The Job controller's purpose is to ensure that a specified number of Pods successfully terminate. It's a fire-and-forget mechanism for work that has a clear end.

How it works

When you create a Job object, the Kubernetes control plane creates one or more Pods based on your template. The Job is considered complete when a specified number of these Pods have terminated with a status of 'Success'. If a Pod fails (e.g., the node it's on reboots or the process crashes), the Job controller will create a new Pod to replace it, up to a configurable limit. This makes your one-off tasks resilient to transient cluster failures.

When to use it

Use a Job for any task that is expected to terminate. Three common scenarios are: first, running a database schema migration before deploying a new version of an application; second, processing a batch of items from a queue until it's empty; third, performing a one-off administrative task across your infrastructure.

When not to use it

Do not use a Job for long-running services like a web server or an API; use a Deployment or StatefulSet for that. For tasks that need to run on a recurring schedule (e.g., nightly backups), use a CronJob, which creates Jobs based on a cron schedule.

One canonical example

A classic example is a Job that calculates pi to a certain number of decimal places. The Job manifest would specify a container image (like perl or python), a command to run the calculation, and a restartPolicy of OnFailure or Never. The Job creates a Pod, the Pod runs the calculation, prints the result, and exits with a success code. The Job then marks itself as complete.