Data Drift vs. Concept Drift: When Models Go Stale

Why it exists

Machine learning models are static snapshots of the world at a specific point in time. But the world is dynamic. Drift is the name for the inevitable divergence between the model's static world and the evolving real world, which causes prediction accuracy to degrade over time.

The mental model

Think of an ML model as a printed map. Drift is what happens when the territory changes—new roads are built, old ones close, and city names are updated. Your old map becomes progressively less useful for navigation because it no longer reflects reality. You need a process to get a new map.

How it works

Drift comes in two main flavors. Concept Drift: This is when the fundamental relationship between the input features and the target variable changes. The 'concept' the model learned is no longer true. For example, in a churn prediction model, the reasons customers leave might change from price sensitivity to poor customer service after a company policy update. The same user data now implies a different outcome. Data Drift: This is when the statistical distribution of the input data itself changes, even if the underlying concept remains the same. For example, a sales forecasting model trained on data from one region is now used globally. The new data has different distributions of income, age, and seasonality, which the model has never seen, causing it to perform poorly.

When to use it

You must plan for drift detection and adaptation in any system where the data generating process is not fixed. This is critical for applications like fraud detection (fraudsters constantly change tactics), demand forecasting (consumer behavior shifts), and recommendation engines (user tastes evolve).

When not to use it

Drift is less of a concern for problems with stable, unchanging rules. A model identifying classic paintings or classifying animal species based on photos deals with concepts that don't change over time. Similarly, a model playing a game with fixed rules like chess doesn't experience concept drift.

One canonical example

A credit scoring model is trained to predict loan defaults. Initially, it learns that having a high number of credit inquiries is a strong predictor of risk. However, a new 'buy now, pay later' service becomes popular, causing many low-risk individuals to have numerous credit inquiries. The meaning of 'high credit inquiries' has changed. This is concept drift, and the model will now incorrectly flag many good applicants as high-risk.