Symptom-based vs cause-based alerting

WHAT THIS TESTS This evaluates whether you can design an alerting policy that minimizes pager noise while reliably catching real user impact, a core SRE skill.

A GOOD ANSWER COVERS Symptom-based alerts trigger on observable user-facing effects: the error rate users experience is high, latency exceeds an SLO, or successful request ratio drops. Cause-based alerts trigger on internal conditions believed to lead to problems: a database replica is down, CPU is saturated, a queue is backing up. The SRE preference is to page humans only on symptoms, because symptoms map directly to whether your service is meeting its SLO and to actual customer pain. Crucially, symptom alerts catch failure modes you never anticipated, since any cause that degrades the user experience surfaces as a symptom, whereas cause alerts only fire for failure modes you predicted and instrumented. Cause signals are still valuable, but they belong on dashboards and as diagnostic context, not as urgent pages.

COMMON WRONG ANSWERS Wanting to page on every internal cause, which floods on-call with alerts that may not affect users, breeds fatigue, and trains people to ignore the pager. Another error is having no cause signals at all, leaving you unable to diagnose once a symptom fires.

LIKELY FOLLOW-UPS How do you avoid a symptom alert that is too coarse to act on? How do error budgets relate to alert thresholds? When is a cause alert worth paging on, such as imminent disk exhaustion? How do you tune for multi-window multi-burn-rate alerting?

ONE CONCRETE EXAMPLE Imagine a redundant three-node cache. A cause alert paging when any single node dies wakes someone even though the cache still serves traffic fine, a non-event. Replace it with a symptom alert: page only when the user-facing error rate or p99 latency breaches the SLO. Now if two nodes fail and users actually suffer, the page fires; if redundancy absorbs a single failure, no one is woken. The node-down condition stays as a dashboard panel and a ticket, giving the engineer the cause once a genuine symptom has paged them.