Run concurrent experiments without interference
WHAT IT TESTS: designing for parallel experiments via layered infrastructure. OUTLINE: independent non-interacting tests can share traffic through orthogonal layers; interacting ones need mutual exclusion in a shared layer.
WHAT IT TESTS: whether you can scale experimentation beyond one test at a time. ANSWER OUTLINE: decide whether the two experiments can interact. If they touch independent surfaces and outcomes, run them in orthogonal layers so each user is independently randomized into both, multiplying capacity. If they could interact, place them in the same layer for mutual exclusion so no user sees both. Use deterministic hashing per layer for independent assignment. Note that orthogonal designs assume no interaction effect, which you should monitor.
Read the original → interview
- #experimentation
- #layered-infrastructure
- #orthogonal-experiments
- #mutual-exclusion
- #experiment-design
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.