Design drift detection and retraining for a CV model.

WHAT THIS TESTS The interviewer wants a closed-loop MLOps design, not just a model, including monitoring under delayed labels, drift detection, and safe automated redeployment.

A GOOD ANSWER COVERS Monitoring: track the distribution of input images and features, the distribution of model predictions and their confidence, and, when ground-truth labels eventually arrive from farmers or experts, actual accuracy. Drift detection: data drift is a shift in inputs, for example new pest species, lighting, or crops, detectable by comparing feature or embedding distributions with tests like population stability index or KS tests; concept drift is a change in the input-to-label relationship, surfacing as falling accuracy or shifting confidence. Because labels are delayed in agriculture, lean on unsupervised input-distribution monitoring and confidence trends as early signals rather than waiting for accuracy. Pipeline: a drift alarm triggers data collection and labeling of recent samples, automated retraining, validation against accuracy and fairness thresholds on a held-out set, then deployment with model and data versioning, a shadow or canary stage comparing new versus old in production, and automatic rollback if metrics regress. A feedback loop captures hard cases, often via active learning, to improve the next round.

COMMON WRONG ANSWERS Monitoring only live accuracy when labels are delayed. Manual, ad hoc retraining with no trigger or validation. No versioning, canary, or rollback, risking a bad model going straight to production.

LIKELY FOLLOW-UPS Distinguishing data from concept drift, choosing drift tests, active learning for labeling, and how often to retrain versus retrain-on-trigger.

ONE CONCRETE EXAMPLE When a new pest appears mid-season, input-distribution monitoring flags unfamiliar images and confidence drops before labels arrive; the system gathers and labels those samples, retrains, validates, and canary-deploys the updated model, rolling back automatically if field accuracy does not improve.