More in MLOps & Infrastructure — page 5
Explain model quantization, its benefits, drawbacks, and validation approach
Tests precision trade-offs in production. Answer: define lowering weights from fp32 to int8/int4; cite memory and latency gains versus accuracy loss; validate with downstream benchmarks and shadow A/B. Red flag: treating as lossless or skipping task metrics.
How would you systematically diagnose high latency in an online inference service?
WHAT IT TESTS: Systems reasoning across serving stack. ANSWER OUTLINE: Check p90/p99 and TTFT to split queuing from compute; inspect queue depth, batch size, GPU, and benchmarks; check cache.
Deploy a trained model as a containerized REST API
This tests practical MLOps fluency. A strong answer covers loading the artifact, wrapping it in a web server, building a Dockerfile, and exposing health and predict endpoints. Red flag: conflating training with serving or omitting resource limits.
Describe the difference between online and batch inference.
WHAT IT TESTS: Your grasp of serving patterns and infra tradeoffs. ANSWER OUTLINE: Online uses autoscaling APIs for millisecond-to-second latency; batch uses scheduled compute for minute-to-hour latency.
Architectural challenges for deploying ML models on resource-constrained edge devices
Tests Edge MLOps architecture under severe constraints. Strong answers hit quantization and delta OTA updates for flaky networks, power-aware scheduling, and closed-loop drift detection.
How would you systematically debug an inference API latency breach?
This tests structured debugging across the full inference stack. A strong answer traces the request path from ingress to GPU, splits TTFT from token-generation latency, inspects queuing and batching, then applies targeted fixes.
Design a system for batch scoring millions of customer records daily
WHAT IT TESTS: Decoupling orchestration, storage, and compute for batch inference with cost/SLA in mind. ANSWER OUTLINE: Shard jobs via scheduler; partition storage; right-size CPU/GPU on spot; retry. RED FLAG: One monolithic VM or real-time APIs for batch.
Where to place feature transformations: client, serving API, or upstream service?
Tests separation of concerns in ML systems. Client causes duplication and skew; serving API couples compute to requests; dedicated service adds a network hop but centralizes logic. Red flag: ignoring training-serving skew.
Describe the difference between online and batch inference.
WHAT IT TESTS: Latency vs throughput trade-offs in serving. ANSWER OUTLINE: Online serves single requests in ms on live endpoints; batch processes data offline with elastic compute. RED FLAG: Treating batch as slow online inference or ignoring SLA and cost.
How would you design safe, automatic schema evolution in CI?
Tests whether you separate schema evolution from semantic validation. Strong answer: versioned data contracts allowing additive enums, unknown-category model buckets, and automated contract negotiation. Red flag: manual allow-lists or disabling validation.
What infrastructure is needed for a Continuous Training pipeline?
Tests event-driven ML system design beyond CI/CD. Strong answers name orchestrators, feature stores, model registries, and validation gates, mapping triggers to retraining and promotion. Red flag: conflating CT with CI/CD or skipping model validation.
How would you implement shadow deployment and which metrics justify promotion?
Tests zero-impact validation when feedback loops are broken. Mirror traffic to a shadow variant, log predictions, and compare latency, errors, and drift against SLAs. Red flag: calling it A/B testing or claiming live business metrics from unserved responses.
What events trigger automatic model retraining beyond code changes?
This tests whether you treat ML pipelines as event-driven systems, not just software CI/CD. A strong answer lists data drift, scheduled cron jobs, production metric degradation, schema changes, and upstream data pipeline completion.
What is a model registry's purpose in CI/CD4ML and its CI/CD interaction?
Tests if you see the model registry as the bridge between experiments and production, not just storage. A strong answer explains how CI publishes validated artifacts and CD consumes versioned models. Red flag: calling it a passive file dump without versioning.
What automated tests belong in CI before deploying a classification model?
WHAT IT TESTS: Distinguishing code tests from ML-specific CI validation. ANSWER OUTLINE: Name data schema checks, performance regression vs baseline, bias audits, and artifact integrity. RED FLAG: Only testing the inference API while ignoring model behavior.
Key differences between traditional and ML CI/CD pipelines?
WHAT IT TESTS: ML CI/CD manages data and model lineage, not just code. ANSWER: Contrast code deploys with data versioning, model registries, and retraining; note holdout eval. RED FLAG: Treating the model as a static binary ignoring data or retraining context.
Design multi-tenant GPU cluster scheduling and preemption policies
Tests ability to design fair GPU scheduling preventing starvation and noisy-neighbor issues. Answer: Kueue for fair-share, namespace quotas with MIG, priority classes with backoff.
Compare Airflow and Kubeflow for ML training pipelines
Tests orchestrator-to-workload fit. Strong answers contrast Airflow's data integration and Python DAGs with Kubeflow's K8s scaling, container reproducibility, and experiment tracking. Red flag: claiming one is always better without stage-specific reasoning.
Design training job submission to a shared Kubernetes cluster
WHAT IT TESTS: Multi-tenant ML infrastructure with usability, fairness, observability. ANSWER OUTLINE: Gateway with artifact caching; namespace quotas; GPU schedulers like Volcano; Prometheus metrics and cost attribution.
Robust checkpointing strategy for multi-day training jobs and seamless resumption
Tests production-grade distributed training reliability. Cover async atomic checkpoints, MTBF-based cadence, tiered storage, and recovery drills. Red flag: blocking synchronous writes that ignore silent corruption or straggler finalization.