Components of a testable A/B hypothesis
WHAT IT TESTS: experimental rigor before launch. OUTLINE: a specific change, a predicted directional effect on one primary metric, a rationale, and a measurable success threshold.
WHAT IT TESTS: whether you can frame an experiment so it can actually be confirmed or refuted. ANSWER OUTLINE: a strong hypothesis names the specific change, the population, a single predicted primary metric with a direction, a causal rationale grounded in user behavior, and a pre-declared decision criterion; it is falsifiable. RED FLAG: a fuzzy statement with no primary metric, no predicted direction, or no threshold, so any outcome can be rationalized after the fact, which is the hallmark of a non-testable hypothesis.
Read the original → interview
- #ab-testing
- #hypothesis
- #experimentation
- #metrics
- #rigor
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.