Intermediate interview questions in MLOps & Infrastructure, page 3
Design a near real-time cost visibility system for ML teams
Tests cost attribution across shared ML infrastructure and streaming pipeline design. Strong answers combine billing exports with resource labels, sub-hour aggregation, and anomaly detection for training spikes.
How do you ensure ML experiment reproducibility beyond random seeds?
Tests system-level reproducibility through data versioning, environment capture, and pipeline automation. Strong answers cover versioned datasets, containerized dependencies, and immutable experiment logs.
How would GDPR requirements influence experiment tracking and model management design?
Immutable data lineage, user exclusion lists, audit logs, versioned explainability.
How would you version control a 50GB dataset in a CI/CD pipeline?
Contrast Git LFS (simple, but 50GB chokes CI clones) with DVC (git metadata plus S3; enables selective pulls and CI cache).
Expose a trained model as a simple web service
Practical MLOps knowledge from model serialization to serving. Package the model into a standard format, containerize it, expose a REST endpoint behind a load balancer, and add monitoring. A bare Flask server without containers or health checks is a red flag.
What is shadow deployment for ML models?
New model receives mirrored live traffic but its predictions are logged, not served; validates real-world behavior and latency without user risk.
Compare Canary and Blue/Green ML deployments and model-specific metrics
Contrast Canary gradual shift vs Blue/Green instant swap; highlight silent failures, data drift, prediction distribution; cite accuracy and calibration.
Design a multi-tenant GPU serving system for hundreds of fine-tuned models
Tests GPU memory tradeoffs versus cold-start latency in multi-tenant serving. Strong answers propose tiered CPU staging, predictive pre-warming, and disaggregated prefill and decode. Red flag: keeping all models GPU-resident or ignoring transfer overhead.
What is a model registry and how does it enable CD?
A registry versions models with metadata, lineage, and stage tags; CD watches stage transitions to trigger deploys.
Explain dynamic batching in inference servers and its trade-off
Dynamic batching launches when a time window or max size is met, improving throughput over static batching, but short ones wait for the slowest.
Concept drift vs data drift in production models
Data drift is a shift in input distribution P(X); concept drift is a shift in the relationship P(Y|X).
Why design ML pipeline steps to be idempotent?
Re-running a step with the same input yields the same result and no duplicate side effects; enables safe retries and backfills.
Compare and contrast Apache Airflow versus Kubeflow Pipelines for ML orchestration
This tests matching orchestrators to ML constraints. A strong answer contrasts Airflow's task scheduling and backfills with Kubeflow's K8s-native GPU scaling, choosing based on team skills.
Design an ML workflow that masks PII from scientists
This tests privacy-preserving pipeline design and least-privilege access for ML teams. Propose automated de-identification before experimentation, restrict re-identification to production jobs, and enforce role-based access with audit logs.
Low GPU utilization on multi-GPU instance: diagnose and right-size
Tests distributed bottleneck triage. Strong answers profile CPU/GPU/disk, compare gradient sync time to compute, validate per-GPU batch size, and check NVLink vs PCIe. Red flag: suggesting more GPUs before ruling out data starvation or all-reduce overhead.
Why systematically track ML experiments and what should you log?
This tests reproducibility mindset over bookkeeping. A strong answer names three motivations—reproducibility, selection, debugging—and three logs: hyperparameters, metrics, and code versions.
Describe a Model Registry and how it differs from versioned storage
It tests governance and lifecycle metadata beyond file storage. A strong answer covers lineage, stage transitions, approval gates, and artifact metadata, contrasting with buckets that only store file versions.
Reproduce a six-month-old model using experiment tracking
Trace code commit, dataset version, feature pipeline, hyperparameters, dependency manifest, and random seeds through a model registry.
Debug sudden model degradation using experiment tracking and model registry
Tests unified use of experiment tracking and registry lineage. Great answers verify the exact production artifact, inspect linked training data and hyperparameters, compare input distributions, and check dependency metadata.
Design a centralized model registry for a large enterprise
Tests ML artifact governance at scale. Strong answers cover immutable versioned artifacts with dependency manifests, a framework-agnostic API, and pluggable deployment targets. Red flag: treating models as opaque files without environment reproducibility.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles