P-Value: Gauging Surprise, Not Certainty
A p-value measures surprise: it's the probability of seeing your results by chance, assuming your change had no effect. It's used in A/B testing to decide if an effect is noise or significant. A small p-value doesn't prove your hypothesis is true.
A p-value is a measure of surprise, not certainty. It answers: 'Assuming my change had zero effect (the null hypothesis), what's the probability of getting a result this extreme just by random chance?' This is used in A/B testing and quantitative fields to evaluate if an outcome is just random variation. The common footgun is thinking a low p-value proves your hypothesis; it only suggests that the 'no effect' scenario is unlikely to explain the data.
Read the original → Wikipedia: P-value
- #statistics
- #ux research
- #a/b testing
- #data science
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.