Why can't you t-test p99 latency, and what's a valid alternative?
WHAT IT TESTS: Knowing percentiles are order statistics, not means. ANSWER OUTLINE: Explain that t-tests target means while p99 variance depends on tail density; propose bootstrap CIs or permutation tests. RED FLAG: Invoking CLT to justify a t-test on p99.
WHAT IT TESTS: Whether you know percentiles are order statistics with non-normal distributions, unlike means. ANSWER OUTLINE: First, note that a t-test compares means and fails for quantiles because tail density is low. Second, describe a bootstrap where you resample each group with replacement, compute p99 each time, and derive a confidence interval for the difference; alternatively, describe a permutation test pooling both groups, shuffling labels, and building a null distribution of percentile differences.
Read the original → allendowney.com
- #statistics
- #ab-testing
- #latency
- #bootstrapping
- #percentiles
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.