How do you determine sample size and duration for an A/B test?
This tests statistical power literacy. A strong answer names baseline rate, MDE, alpha, and beta; explains the duration versus sensitivity trade-off; and notes traffic allocation. A red flag is ignoring power or stopping early when results look significant.
This tests whether you can design a valid experiment rather than blindly run tests. A strong answer walks through a power analysis using baseline conversion rate, Minimum Detectable Effect, significance threshold alpha, and desired power one-minus-beta. It then connects sample size to duration via daily traffic, covers the trade-off between smaller MDE and longer runtimes, and calls out risks like seasonality and peeking. A red flag is treating sample size as an afterthought or suggesting you can stop the moment a p-value dips below 5 percent.
Read the original → cxl.com
- #ab testing
- #statistics
- #experimentation
- #sample size
- #power analysis
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.