Why use latency percentiles over the average?

WHAT THIS TESTS This evaluates whether you understand latency as a distribution rather than a single number, and whether you can reason about when to watch the extreme tail versus a less extreme percentile.

A GOOD ANSWER COVERS Explain why averages mislead. Latency distributions are typically right-skewed and sometimes bimodal, so a handful of very slow requests, or many fast ones, pull the mean away from typical experience. The average can sit comfortably under target while a meaningful share of users hit slow responses, because the metric hides the spread. Percentiles fix this by describing the actual experienced latency: p50 is the median typical case, p95 and p99 expose the slow tail that frustrates users. Then address p99 versus p95. Prefer p99 when the tail is disproportionately important: when even one percent of slow requests carries high business cost, when requests fan out to many backends so a single user request waits on the slowest of many calls and tail latency compounds, or for premium or critical flows where worst-case experience matters. Prefer p95 when p99 is too noisy or dominated by unavoidable rare outliers, to keep alerts actionable.

COMMON WRONG ANSWERS Defending the average as sufficient. Believing p99 is universally superior and should always replace p95. Forgetting fan-out amplification. Confusing percentile with percentage of capacity. Treating a single p99 number as meaningful without a window or sample size.

LIKELY FOLLOW-UPS How does fan-out turn p99 of a backend into typical user experience? Why can p99 be noisy? How do you set the alerting threshold? What about p999?

ONE CONCRETE EXAMPLE A dashboard shows mean latency of 80 milliseconds, looking fine, but p99 is 2 seconds. A page assembles results from twenty backend calls, so the slowest of twenty per request dominates user-perceived latency; with a one percent tail, most page loads hit at least one slow call. Here alerting on p99 is essential because the average completely hid an experience that affected nearly every user, while fan-out made the tail the common case.