Sample Ratio Mismatch (SRM): When Your A/B Test Is Broken
Sample Ratio Mismatch (SRM) means your A/B test's traffic split is broken, violating random assignment. For example, a 50/50 split results in a statistically significant imbalance.
Sample Ratio Mismatch (SRM) means your A/B test's traffic distribution is broken. The actual number of users in each variation doesn't match your intended split, indicating a fundamental bias in traffic assignment. This occurs when a test configured for a 50/50 split shows a statistically significant deviation, often due to bugs affecting only certain browsers or devices. The biggest mistake is ignoring an SRM warning and trusting the results; the underlying bias invalidates your conclusions and can lead to poor business decisions.
Read the original → docs.kameleoon.com
- #a/b testing
- #analytics
- #statistics
- #experimentation
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.