Handle interaction effects on a shared page
WHAT IT TESTS: reasoning about interaction effects and mitigation. OUTLINE: combined variants may produce effects neither has alone; use mutual exclusion for likely interactions, orthogonal designs with interaction monitoring otherwise.
WHAT IT TESTS: whether you understand interaction effects and how to design around them. ANSWER OUTLINE: when two experiments change the same page, their variants can combine in ways that produce a joint effect different from the sum of individual effects, confounding each test. Mitigations: place likely-interacting experiments in the same layer for mutual exclusion so no user gets both; if running orthogonally, monitor for interactions by slicing each experiment's results by the other's variant, and increase sample size to detect them.
Read the original → interview
- #interaction-effects
- #experimentation
- #mutual-exclusion
- #factorial-design
- #experiment-design
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.