Automating actions on error budget burn
Whether you can operationalize error budgets with automation.
Use multi-window burn-rate alerts; on fast burn, auto-trigger actions like halting deploys, rolling back, or scaling, with escalating tiers.
What's really being asked
Whether you can turn the error budget from a reporting metric into a live control signal that drives alerting and automated remediation, using burn rate rather than static thresholds.
The full answer
The key concept is burn rate, the speed at which the budget is being consumed relative to the SLO window. A burn rate of one would exhaust the budget exactly at the end of the window; higher rates exhaust it faster. Use multi-window, multi-burn-rate alerting: a high burn rate over a short window (for example a fast burn over minutes) pages immediately, while a moderate burn over a longer window opens a ticket, balancing fast detection against false alarms. For automation, map burn-rate tiers to actions. On a rapid burn shortly after a deploy, automatically freeze further deploys and trigger a rollback of the most recent release, since recent changes are the likeliest cause. Other automated responses include scaling out capacity if the burn is load-driven, shedding non-critical load, or switching to a degraded fallback mode. Escalate tiers: warn, then block deploys, then page and roll back. Every automated action must be logged, reversible where possible, and capable of escalating to a human.
The mistakes people make
Alerting on absolute budget remaining rather than burn rate, which fires too late. Single-window alerts that are either too noisy or too slow. Automating irreversible or risky actions with no guardrails, or coupling automation so tightly that a metric glitch triggers a needless rollback.
What usually comes next
How do you pick burn-rate thresholds and windows? When should burn trigger automation versus just paging? How do you prevent a noisy SLI from triggering false rollbacks?
A concrete example
A service deploys at noon; by 12:05 the burn rate spikes to 14x over a five-minute window. The system pages on-call, automatically blocks further deploys, and rolls back the noon release. The burn rate drops, confirming the deploy was the cause, and the rollback is logged for the postmortem.
Interview question
Why do burn-rate alerts use multiple time windows rather than alerting purely on remaining budget?
- a.Burn rate is only relevant for stateless services
- b.They detect fast burns urgently while keeping slow burns low-noise, instead of alerting only after the budget is goneCorrect
- c.Multiple windows make dashboards more colorful
- d.Remaining budget cannot be measured in real time
Why? this is the answer
Multi-window, multi-burn-rate alerting catches rapid consumption quickly and treats slow consumption as low-priority, balancing detection speed and noise. Alerting on remaining budget alone fires only once damage is nearly complete.
Just read this? Test yourself on what you have been reading.
- #sre
- #error-budget
- #burn-rate
- #alerting
- #automation
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on sre — each one lists the topics its interview covers.
See open roles