Kubernetes Probes: Liveness, Readiness, and Startup

WHY IT EXISTS: Without probes, Kubernetes only knows if a container process has started. It can't tell if the application inside the container is frozen, deadlocked, or still loading data. Probes give Kubernetes a window into the application's internal state, enabling automated recovery and traffic management.

THE MENTAL MODEL: Think of a restaurant chef. A LIVENESS probe is a pulse check: if the chef is unconscious, they're replaced (container restart). A READINESS probe asks if they can take a new order: if they're swamped, you stop sending tickets (remove from service endpoints). A STARTUP probe is for the initial setup, ensuring the kitchen is ready before the first order arrives, preventing the other checks from failing prematurely.

HOW IT WORKS: The kubelet on each node periodically runs a check against your container. This check can be an HTTP GET request to a health endpoint, an attempt to open a TCP socket on a port, or a command executed inside the container. Based on the probe type, Kubernetes takes different actions. LIVENESS: If a liveness probe fails a configured number of times, the kubelet kills the container and restarts it based on the pod's restart policy. This is for recovering from unrecoverable application states. READINESS: If a readiness probe fails, the pod's IP address is removed from the endpoints of all matching Services. It won't receive new traffic until the probe succeeds again. The container is not restarted. STARTUP: This probe runs at the beginning of the container's life. If it fails beyond its threshold, the container is restarted. If it succeeds, the liveness and readiness probes take over. This protects slow-starting applications from being killed by a liveness probe that expects a fast startup.

WHEN TO USE IT: Use probes for any long-running service. Use a liveness probe to recover from deadlocks. Use a readiness probe when your app needs time to warm up, load data, or can become temporarily overloaded. Use a startup probe for applications with long, variable initialization times, like a Java application warming up its JVM.

WHEN NOT TO USE IT: Avoid liveness probes if a restart would cause data loss or if the application is designed to run to completion and exit. The biggest footgun is creating a liveness probe that depends on external services (like a database). If that dependency becomes slow, your application might enter a crash loop, even though the app itself is fine.

ONE CANONICAL EXAMPLE: A web application has a liveness probe at /healthz that simply returns a 200 OK to confirm the web server is running. It has a separate readiness probe at /ready that checks its database connection. If the database connection is lost, the readiness probe fails, Kubernetes stops sending traffic to that pod, but the liveness probe continues to pass, preventing a pointless restart while the app waits for the database to recover.