Explain statistical power and respond to extending a null A/B test

This tests statistical power and p-hacking judgment. A strong answer defines power as detecting a true effect, rejects extending the test to chase significance, and requires pre-registered sample sizes. Agreeing to run until it hits significance is a red flag.
This tests whether you understand statistical power as the probability of detecting a true effect and recognize that extending a test after a null result is p-hacking. A strong answer defines power as one minus beta, explains that it is fixed during design by sample size and minimum detectable effect, refuses to extend since that invalidates p-values, and suggests reporting the effect size with a confidence interval then planning a new study. The interviewer listens for candidates who agree to extend the test or confuse power with sample size.
Read the original → cxl.com
- #statistics
- #ab-testing
- #experimentation
- #stakeholder-management
- #power-analysis
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.