Regression to the Mean: Why Outliers Settle Down

Why it exists

We constantly need to distinguish between a real change in a system and a temporary, random fluctuation. Without understanding regression to the mean, we risk making bad decisions based on outlier data, rewarding luck as if it were skill and punishing misfortune as if it were incompetence.

The mental model

Think of any performance as a simple equation: Performance = True Skill + Random Chance. A record-breaking sales quarter is high True Skill plus a lot of good Random Chance (luck). The next quarter, the True Skill is still there, but the Random Chance is likely to be closer to zero, or even negative. The result will naturally "regress" toward the average performance dictated by True Skill alone.

How it works

When you observe a random variable, an extreme outcome is, by definition, rare. For that extreme outcome to occur again, the same rare circumstances (the "luck") would need to happen twice in a row, which is statistically even less likely. Therefore, the next measurement is more probable to be closer to the mean, which is the most common outcome. This applies to single data points and also to groups selected specifically because of their extreme initial scores.

When to use it

Use this concept to temper your reactions to extreme data. First, when analyzing A/B test results, especially early on; a huge initial win for a variant might settle down to a more modest improvement. Second, in performance management; a star engineer's most productive sprint and a struggling engineer's worst are both unlikely to be repeated immediately. Third, when evaluating any intervention that follows an extreme event, like a new policy after a security breach or a marketing push after a sales slump. The situation was likely to improve anyway.

When not to use it

Do not use it to dismiss all changes. If a result is consistently above or below the previous mean over many measurements, it likely indicates a real, fundamental shift in the underlying system, not just a random fluctuation. Regression to the mean describes the behavior of short-term outliers, not the establishment of a new long-term trend.

One canonical example

In pilot training, instructors observed that praising a pilot for an exceptionally smooth landing was often followed by a worse landing. Conversely, yelling at a pilot for a very rough landing was followed by a better one. They concluded that criticism was effective and praise was not. In reality, they were just observing regression to the mean. The exceptionally good and bad landings were outliers, and subsequent performances were simply regressing back toward the pilots' average skill level.