Concept drift vs data drift in production models

WHAT THIS TESTS Whether you can distinguish the two main causes of model degradation and design monitoring that actually detects each, including the uncomfortable fact that one of them cannot be seen from inputs alone.

A GOOD ANSWER COVERS Data drift, sometimes called covariate shift, is a change in the distribution of the inputs P(X) while the underlying relationship between inputs and the target stays the same. Concept drift is a change in P(Y|X), the actual relationship the model was trained to approximate, which can happen even when the input distribution is unchanged. The two are independent and either can occur without the other. Detecting data drift is comparatively easy because you only need the inputs: compare recent feature distributions against a reference using statistical tests or population stability metrics. Detecting concept drift is harder because the inputs can look identical while the correct answer has changed, so you fundamentally need ground-truth labels and must track predictive performance or error over time, often with a delay while labels arrive.

COMMON WRONG ANSWERS Claiming input distribution monitoring alone catches concept drift is wrong, because P(Y|X) can shift with P(X) unchanged. Treating the two terms as synonyms, or assuming retraining fixes both equally without diagnosing which occurred, is also weak.

LIKELY FOLLOW-UPS How do you monitor when labels are delayed or expensive? What proxy signals hint at concept drift before labels arrive? How do you decide between retraining and a full model redesign?

ONE CONCRETE EXAMPLE Data drift without concept drift: you launch in a new country, so feature values shift, but a heavier user still predicts the same purchase pattern. Concept drift without data drift: during a recession, applicant features look statistically normal, yet the same profiles now default far more, so credit-risk error climbs even though input monitors show nothing. The first is caught by feature monitors; the second only by label-based error tracking.