Steady State Hypothesis: The Core of Chaos Engineering

WHY IT EXISTS Distributed systems are complex and can fail in unpredictable ways. To build confidence in their ability to withstand turbulence, we need a scientific way to uncover weaknesses before they impact customers. The steady state hypothesis provides the formal framework for these experiments.

THE MENTAL MODEL Think of the steady state hypothesis like a doctor performing a cardiac stress test. The hypothesis is: "This patient's vital signs will remain in a healthy range while on the treadmill." The experiment is the treadmill itself. If the patient's vitals go haywire, the doctor has disproven the hypothesis and found a weakness to address.

HOW IT WORKS Chaos Engineering experiments follow a clear process based on this hypothesis. First, you define your system's 'steady state' using measurable, user-facing outputs that indicate normal behavior, like system throughput, error rates, or latency percentiles. Second, you hypothesize that this steady state will continue in both a control group and an experimental group. Third, you introduce a variable that reflects a real-world event, like a server crash or severed network connection, to the experimental group. Finally, you try to disprove the hypothesis by looking for a difference between the two groups. If the steady state breaks, you've found a systemic weakness.

WHEN TO USE IT Use this hypothesis as the foundation for any Chaos Engineering experiment. It is the primary tool for verifying that a system's resilience mechanisms—such as fallbacks, retries, and circuit breakers—actually work under realistic failure conditions. The harder it is to disrupt the steady state, the more confidence you have in your system.

WHEN NOT TO USE IT This approach is for verifying that a system works as a whole, not how its individual parts function internally. Avoid it for unit testing or debugging a specific component's logic. The focus is on the observable, external behavior of the system, not its internal state.

ONE CANONICAL EXAMPLE An e-commerce platform's SRE team hypothesizes that their p99 latency will remain below 400ms and the checkout error rate below 0.1% (the steady state) even if a primary database fails over. They run an experiment that forces a database failover. They observe that latency spikes to 1500ms for 30 seconds. The hypothesis is disproven, revealing a weakness in connection handling during failover that needs to be fixed.