Triaging a 5xx error spike
Calm, structured incident triage.
confirm impact and scope, declare and assign roles, check recent changes, stop the bleeding before root cause.
diving into code fixes before assessing blast radius or mitigating.
WHAT THIS TESTS Whether you have a repeatable triage method and prioritize restoring service over satisfying curiosity. Interviewers want to see calm sequencing under pressure, not heroics.
A GOOD ANSWER COVERS Start by confirming the signal is real and not a monitoring artifact, then quantify impact: what fraction of requests are 5xx, which endpoints, which regions or customers, and whether it is climbing. If impact is material, declare an incident and assign roles so coordination and communication are owned. Next, correlate timing with recent changes: deploys, config flags, dependency outages, or traffic surges, since most incidents follow a change. Then mitigate first to stop user pain, typically a rollback, feature flag off, or load shedding, and only afterward dig into root cause.
COMMON WRONG ANSWERS Opening an IDE to fix the bug before knowing blast radius. Skipping the impact assessment, so the response is over or under scaled. Forgetting to communicate status to stakeholders.
LIKELY FOLLOW-UPS When do you escalate or page secondary on-call? How do you decide rollback versus forward fix? What do you log for the post-mortem timeline?
ONE CONCRETE EXAMPLE A dashboard shows checkout 5xx jumping from 0.1 percent to 12 percent at 14:02. You confirm it across two regions, declare a SEV2, and take comms. The error onset matches a 14:00 deploy of the payments service. Rather than read the diff, you roll that deploy back; error rate returns to baseline within four minutes. Only then do you pull traces and logs to find the bad config that caused it, capturing the timeline as you go for the review.
Read the original → sre.google
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.