tezvyn:

Explain statistical power and respond to extending a null A/B test

Source: cxl.comadvanced

This tests statistical power and p-hacking judgment. A strong answer defines power as detecting a true effect, rejects extending the test to chase significance, and requires pre-registered sample sizes. Agreeing to run until it hits significance is a red flag.

This tests whether you understand statistical power as the probability of detecting a true effect and recognize that extending a test after a null result is p-hacking. A strong answer defines power as one minus beta, explains that it is fixed during design by sample size and minimum detectable effect, refuses to extend since that invalidates p-values, and suggests reporting the effect size with a confidence interval then planning a new study. The interviewer listens for candidates who agree to extend the test or confuse power with sample size.

Read the original → cxl.com

Get five bites like this every day.

Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.

Explain statistical power and respond to extending a null A/B test · Tezvyn