Deploy a trained model as a containerized REST API

WHAT THIS TESTS: This question evaluates whether you understand the minimal viable path from a trained artifact to a live inference service. At the senior level, interviewers care less about memorizing framework syntax and more about your awareness of the serving lifecycle: loading, wrapping, containerizing, and exposing. It reveals whether you conflate training infrastructure with prediction infrastructure.

A GOOD ANSWER COVERS: A good answer hits four things in order. First, load the serialized model into memory once at startup, not per request, to avoid catastrophic latency. Second, wrap the model in a lightweight web framework such as FastAPI or Flask and define at least two routes: a health check endpoint for orchestrators and a predict endpoint that accepts structured input and returns structured output. Third, write a Dockerfile that uses a slim base image, installs only the inference dependencies, copies the model artifact and application code, exposes a port, and sets a production server command. Fourth, run the container with a production-grade HTTP server like Uvicorn or Gunicorn rather than the framework's development reloader.

COMMON WRONG ANSWERS: Common wrong answers include loading the model inside the predict function so it reloads on every call, using a development server for production traffic, omitting a health endpoint entirely, or describing a batch prediction pipeline instead of a real-time REST API. Another red flag is ignoring input validation and returning raw tensors without serialization.

LIKELY FOLLOW-UPS: Interviewers often push deeper with questions like these: how would you scale this horizontally under load; how do you handle GPU access inside the container; what is your strategy for A/B testing model versions; how do you monitor prediction latency and error rates; and how would you validate incoming request schemas before they reach the model.

ONE CONCRETE EXAMPLE: Imagine a scikit-learn model saved as a joblib file. You write a FastAPI application that loads the joblib artifact when the module initializes, defines a Pydantic model for input validation, and exposes POST /predict and GET /health. Your Dockerfile starts from python:3.11-slim, installs scikit-learn and fastapi via a requirements.txt, copies the joblib file and main.py, and sets the command to uvicorn main:app --host 0.0.0.0 --port 8080. You build the image, run it mapping port 8080, and send JSON requests to receive predictions.