Escalation Policy: When to Stop Shipping and Start Fixing
An escalation policy is a pre-agreed plan for when to divert engineers from feature work to fix reliability. When a service's error budget burns too fast, the policy's thresholds trigger specific actions. The footgun is thinking a quick rollback is enough.
Why it exists
To create a formal agreement between development and operations on when to prioritize reliability over new features. Without a policy, teams can endlessly debate whether to fix an unstable service or continue shipping, leading to inconsistent responses and on-call burnout.
The mental model
Think of it as a circuit breaker for feature development. When a service's reliability, measured by its error budget, drops below a certain point, the policy trips. This automatically redirects engineering focus from building new things to stabilizing what already exists, based on pre-agreed rules.
How it works
An escalation policy defines a series of thresholds and corresponding actions based on the rate of error budget consumption for a given Service Level Objective (SLO). When a threshold is crossed, a pre-defined action is triggered. The key is that "resolving" the issue requires more than a temporary fix. To consider a service back in SLO, the team must either fix the entire class of problem (e.g., make bad config pushes impossible), automate the remediation, or prove the issue is extremely unlikely to recur. A plan for future manual fixes is not sufficient.
When to use it
Use an escalation policy to formalize the trade-off between development velocity and service reliability. It is especially valuable for services where time-to-market is important and some instability is acceptable. It provides a clear, data-driven framework for making decisions that would otherwise be purely political or emotional.
When not to use it
A policy that prioritizes velocity is a poor fit for services with extremely high availability targets, like 99.99% or higher. For these critical systems, the tolerance for error is so low that any SLO violation likely requires immediate, all-hands attention, rather than a tiered escalation. The policy must be tailored to the service's business context.
One canonical example
A Google SRE team's policy states that if a service burns nine hours of its error budget within a single hour, an alert pages the on-call SRE for immediate investigation. If the service burns one week of error budget over the course of a week, a ticket is automatically filed for non-urgent investigation. This tiered response matches the severity of the budget burn to the urgency of the engineering response.
Interview question
According to the card, for which type of service would an escalation policy be LEAST appropriate?
- a.A rapidly evolving consumer application prioritizing quick feature releases over perfect stability.
- b.A critical financial service aiming for 99.999% uptime where any outage is unacceptable.Correct
- c.A backend microservice experiencing inconsistent responses to recurring reliability issues.
- d.A system with a tiered error budget, where minor burns trigger non-urgent investigations.
Why? this is the answer
The card explicitly states that a policy prioritizing velocity is a poor fit for services with extremely high availability targets (e.g., 99.99% or higher), as any violation requires immediate, all-hands attention. The other options describe scenarios where an escalation policy is either beneficial or designed to operate.
Just read this? Test yourself on what you have been reading.
Read the original → cloud.google.com
- #sre
- #monitoring
- #slo
- #on-call
- #reliability
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on sre — each one lists the topics its interview covers.
See open roles