Batch prediction versus online real-time prediction

WHAT THIS TESTS This probes whether you understand that how a model is served is a design choice driven by latency, freshness, and cost, not a one-size-fits-all decision. It is foundational MLOps literacy.

A GOOD ANSWER COVERS Batch prediction runs the model over a large set of inputs on a schedule, for example hourly or nightly, writing results to a database or file for later lookup. It optimizes for throughput and cost efficiency, tolerates high latency, and is simple to operate. Online prediction serves one request at a time synchronously through an API, optimizing for low latency, often tens of milliseconds, and requires always-on, scalable infrastructure. The key tradeoffs are freshness versus cost: batch gives cheaper, stale-by-design results, while online gives fresh, per-request results at higher operational cost and complexity. Make clear the model itself can be identical; only the serving mode differs.

COMMON WRONG ANSWERS Confusing batch prediction with batch or offline training. Claiming online prediction is always better, ignoring that many problems do not need fresh-per-request scoring. Failing to give concrete, distinct use cases.

LIKELY FOLLOW-UPS How would you handle a case that needs near-real-time but not strict real-time? Can you precompute batch results and serve them online? How does each affect cost?

ONE CONCRETE EXAMPLE For batch: a subscription company scores every customer's churn risk overnight and loads the scores into the CRM so account managers see updated risk each morning. For online: a payments platform scores each transaction for fraud at the moment of checkout, returning an approve-or-decline decision in under a hundred milliseconds because the user is waiting. The first tolerates a day of staleness; the second cannot wait at all.