How would you structure a backend architecture A/B test and define metrics?
This tests causal inference rigor for infrastructure changes. A strong answer covers sticky user routing, controlling for geography and time, and paired primary metrics like P99 latency and error rate.
This tests controlled experiment design for backend systems where shared resources and skewed latency obscure causality. A strong answer covers four elements: sticky user routing to prevent cross-contamination; stratification by region or device to balance covariates; a parallel canary or holdback to isolate deployment noise; and paired metrics such as P99 latency, error rate, and a UX guardrail like conversion. Red flags include ignoring SUTVA violations, using mean instead of tail latencies, or skipping a rollback rule.
Read the original → Wikipedia: A/B testing
- #ab testing
- #backend architecture
- #experimental design
- #latency
- #infrastructure
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.