Differences between monitoring a traditional REST API and a production ML model

WHAT THIS TESTS: This question probes whether you recognize that machine learning systems are fundamentally probabilistic and data-dependent, meaning they can fail silently when input distributions shift even if infrastructure metrics look perfectly healthy. Interviewers want to see that you distinguish between operational health and model health, and that you understand the statistical nature of ML monitoring compared to deterministic software checks.

A GOOD ANSWER COVERS: Four distinctions in order. First, deterministic versus probabilistic behavior. Traditional REST APIs execute explicit code rules, so monitoring focuses on latency, throughput, error rates, and uptime. Machine learning models produce outputs learned from training data, so identical infrastructure metrics do not guarantee correct predictions. Second, data-centric signals. You should name specific ML monitoring signals such as data drift, prediction drift, data quality degradation, feature attribution drift, and training-serving skew. Third, baseline comparison methodology. Explain that ML monitoring relies on statistical tests or distance scores comparing production inference distributions against reference baselines like training or validation data, rather than simple threshold checks on CPU or memory. Fourth, ground truth latency. In traditional services, a five hundred error is immediate and obvious. In ML, true model performance often requires ground truth labels that arrive days or weeks later, creating a delayed feedback loop that standard API dashboards cannot expose.

COMMON WRONG ANSWERS: The biggest red flag is answering with only standard SRE metrics like request latency and availability while completely ignoring data distributions. Another weak pattern is mentioning drift vaguely without explaining what distributions are being compared or why statistical tests are needed. Equating model staleness with a standard software bug is also incorrect; models degrade gradually as the world changes, not because of a single bad deploy.

LIKELY FOLLOW-UPS: Expect the interviewer to ask how you would detect data drift without ground truth labels, or how you would handle a scenario where infrastructure metrics are green but business metrics are dropping. They may also probe whether you have implemented automated retraining pipelines triggered by drift thresholds, or how you would monitor a model deployed on edge devices with limited telemetry. Be ready to discuss which statistical tests you would use for continuous versus categorical features.

ONE CONCRETE EXAMPLE: Suppose you deploy a fraud detection model. A traditional monitoring dashboard shows ninety nine point nine percent uptime and two hundred millisecond p99 latency. However, over two weeks, a new merchant category grows from one percent to forty percent of transaction volume. The model was never trained on this category, so input feature distributions shift significantly. Without ML-specific monitoring like data drift alerts comparing production features to the training baseline, this silent degradation goes unnoticed until chargeback rates spike three weeks later when ground truth labels finally arrive.

Source: learn.microsoft.com