Why prefer median and p95 over mean for API latency?

WHAT THIS TESTS: This question tests whether you understand that latency data is not normally distributed and that averaging it produces a meaningless number. Interviewers want to see statistical intuition about skewed distributions, tail latency, and user-centric observability. They also want to know if you can articulate why the worst experiences matter as much as the typical ones.

A GOOD ANSWER COVERS: First, state that API response times follow a right-skewed distribution with a hard floor near zero and a long tail caused by outliers. Second, explain that the median or p50 represents the typical user experience because half of all requests are faster and half are slower, making it robust to outliers. Third, explain that the p95 or higher percentiles capture tail latency, which is critical because those slow requests represent real users suffering timeouts or degraded service. Fourth, note that the mean is sensitive to extreme values: a few multi-second outliers can pull the average up and make a struggling system look healthy. Fifth, mention that SLOs and alerting should be based on percentiles rather than means.

COMMON WRONG ANSWERS: Claiming that mean is acceptable because it is easy to compute. Saying that median and p95 are only for large scale systems. Failing to mention that outliers come from real phenomena like garbage collection, cache misses, lock contention, or retries. Arguing that mean plus standard deviation is enough, which assumes a normal distribution that latency rarely follows. Giving a scenario where the mean is misleading but not explaining why the median would differ.

LIKELY FOLLOW-UPS: How would you implement percentile calculation efficiently in a high-throughput service? What is the difference between service time and response time? At what percentile does latency stop mattering for user experience? How do you handle the coordinated omission problem when measuring latency? What histogram aggregation strategy would you use for metrics collection?

ONE CONCRETE EXAMPLE: Imagine an API where ninety-nine percent of requests complete in ten milliseconds, but one percent hit a cold cache and take two seconds. The mean would report roughly twenty-nine milliseconds, suggesting excellent performance. Meanwhile the p50 is ten milliseconds and the p95 is two seconds. If you relied on the mean, you would miss that one in twenty users is getting a terrible experience and might abandon the request.