Hypothesis Testing: Is Your Data Signal or Noise?
Hypothesis testing is a courtroom trial for your data: you assume a default 'null hypothesis' is true until your data provides enough evidence to reject it. It's used in A/B tests to validate changes.
Think of hypothesis testing as a courtroom trial for your data. You start by assuming a 'null hypothesis' (the status quo) is true, then calculate how likely your observed data is under that assumption. If it's very unlikely (below a p-value threshold), you reject the null. This is crucial for A/B testing to confirm if a new feature truly improved metrics. The common footgun is misinterpreting the p-value: it's the probability of your data given the null, not the probability the null is false.
Read the original → Wikipedia: Statistical hypothesis testing
- #statistics
- #a/b testing
- #data analysis
- #metrics
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.