What is shadow deployment for ML models?

WHAT THIS TESTS Whether you can validate a new model under real production conditions without exposing users to its mistakes, and whether you understand the difference between shadow mode and other rollout patterns.

A GOOD ANSWER COVERS Shadow mode mirrors live production traffic to a new candidate model running in parallel with the current model. The candidate produces predictions for every request, but those predictions are only logged, never returned to the caller. The production model continues to serve all users. You then compare the shadow predictions against the live model and, where available, against eventual ground truth labels. The benefits are concrete: you observe behavior on the true input distribution rather than a sampled or synthetic test set, you measure real latency and resource consumption under production load, and you catch serving skew, missing features, or schema mismatches before any user is affected. Technical requirements include a traffic mirroring or duplication layer, asynchronous non-blocking inference so the shadow path never adds latency to the live response, durable logging of both models' outputs keyed by request id, and an offline comparison and metrics harness.

COMMON WRONG ANSWERS Describing canary or A/B testing, where the new model actually serves some real users, defeats the entire point of shadow mode being risk-free. Another mistake is running the shadow model synchronously in the request path, which adds latency and can degrade the live experience.

LIKELY FOLLOW-UPS How do you handle non-idempotent side effects such as writes or external API calls from the shadow model? How long do you run shadow mode before promoting? How do you compare when ground truth arrives much later than the prediction?

ONE CONCRETE EXAMPLE A fraud-scoring service mirrors every transaction to a new model. For two weeks its scores are logged but the old model still decides approvals. Engineers discover the new model times out on large merchant batches and disagrees on 4 percent of high-value cases, so they fix the timeout before promotion.