tezvyn:

Error budget policy across dependent microservices?

AI-drafted, machine-checkedSource: interviewadvanced
WHAT IT TESTS

SLO design in a dependency graph.

OUTLINE

set per-service SLOs but anchor on user-facing journey SLOs, budget for dependency error via the multiplication of availabilities, attribute downstream-caused failures correctly, and use…

WHAT THIS TESTS This advanced question evaluates whether you can design error budgets for an interdependent system, handling the math of composed availability and the politics of fair attribution across teams.

A GOOD ANSWER COVERS Anchor on user-facing SLOs for critical journeys, since users only care about end-to-end behavior. Decompose those into per-service SLOs, recognizing that along a serial dependency chain availabilities multiply, so if a request needs three services in sequence, each must be more reliable than the end-to-end target. Make dependencies explicit and require critical dependencies to commit to SLOs at least as strict as their dependents need. Crucially, design attribution: when an upstream service breaches its SLO purely because a downstream dependency failed, the budget burn should be charged to the downstream service and team, so the policy creates correct incentives rather than punishing the victim. Provide mechanisms to exclude dependency-caused errors from the upstream team's budget while counting them against the downstream team's. Each team's error budget policy then governs its own launch freezes, but cross-team escalation handles shared-fate incidents. Account for fan-out, retries, and graceful degradation, which can soften or amplify dependency impact.

COMMON WRONG ANSWERS Treating microservices as independent and ignoring composed availability. Failing to attribute downstream failures, so upstream teams are penalized unfairly. Naively summing SLOs. Forgetting that retries and fallbacks change the effective dependency reliability.

LIKELY FOLLOW-UPS How do you measure which service caused a given failure? How do graceful degradation or caching change the budget math? What about cyclic or fan-out dependencies? How do you arbitrate cross-team disputes over attribution?

ONE CONCRETE EXAMPLE A checkout journey depends serially on auth and payments. The end-to-end SLO is 99.9 percent, so auth and payments each target around 99.95 percent to leave room. When payments has an outage and checkout's availability drops, the monitoring attributes those specific failed requests to payments. Checkout's error budget is shielded from that burn, payments' budget absorbs it and triggers its freeze policy, and the payments team owns the fix, producing fair, incentive-aligned accountability.

Read the original → cloud.google.com

Get five bites like this every day.

Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.