Interview questions in MLOps & Infrastructure, page 5
What is a feature store and how does it prevent training-serving skew?
This tests training-serving consistency via centralized feature management. Covers offline batch storage, online serving, shared transformations, and alternatives like ad-hoc ETL. A red flag is calling it just a database and ignoring point-in-time correctness.
What trade-offs decide managed ML platforms versus open-source Kubernetes?
Weigh total cost plus hidden engineering headcount, lock-in vs flexibility, and audit feature gaps.
How would you design a reproducible ML training pipeline?
Tests if you can version ML's three moving parts: code, data, and environment. Good answers cover Git for code, DVC or lakehouse versioning for data, and Docker plus locked dependencies for environments.
Serving for online and batch predictions
A low-latency online path (synchronous, autoscaled, real-time features) and a high-throughput batch path (parallel, cost-optimized, large jobs) sharing one model artifact and feature definitions.
Design a multi-tenant ML platform with isolation, security, and cost attribution
Tests mapping tenancy to compute, network, and identity primitives. Strong answers compare hard vs soft isolation, use namespaces or node pools with network policies and IAM, and enforce chargeback via resource quotas and labels.
Propose an architectural solution for contended GPU training resources
Tests multi-tenant GPU scheduling design at scale. Great answers tier jobs by checkpointability, apply quota-based preemption, mix spot and on-demand instances, and use MIG or time-slicing to bin-pack. Red flag: buying GPUs without scheduling logic.
How do you give read-only access to a shared cloud storage bucket?
Bind an IAM role with read permissions to the team at the bucket level, avoid object-level ACLs, and mount read-only on training VMs.
Why was this customer denied: global or local explanation?
This tests matching questions to explanation scope. Global methods show overall behavior; local methods explain one prediction. Specific denials need local methods like SHAP. A red flag is using global summaries like permutation importance or PDPs for a case.
Describe securing an automated ML pipeline and CI/CD integration points
Tests ML supply-chain depth versus bolt-on appsec. Strong answers stage checks across
How would you programmatically monitor a deployed model for demographic bias?
Tests operationalizing fairness beyond static audits. Track group metrics like parity and equalized odds; slice by protected attributes; alert on drift; route violations to review. Red flag: treating fairness as a one-time check versus continuous monitoring.
What is the wrong and right way to manage ML database secrets?
This tests secret management hygiene for ML pipelines. A strong answer rejects hardcoded secrets and env vars, then proposes AWS Secrets Manager with IAM retrieval, TLS, caching, and rotation. A red flag is suggesting .env files, ConfigMaps, or CLI arguments.
Design a cryptographically verifiable ML audit trail from dataset to deployment
Tests cryptographic provenance and tamper-evident ML pipelines. Strong answers cover content-addressed datasets, signed training logs linking code and hyperparameters to model hashes, and deployment signature checks.
Design a defense-in-depth strategy against adversarial evasion on a deployed image classifier
Proactive: adversarial training, preprocessing, ensembles.
How do you attribute cloud costs to ML projects and implement tagging?
Tests knowledge of resource tagging for cost attribution. A strong answer names provider-specific tags or labels, embeds them in infrastructure-as-code, and activates cost allocation reports.
Differences between on-demand, reserved, and spot EC2 instances?
Tests cost-reliability-commitment tradeoffs for ML infrastructure. Good answers map on-demand to experiments, reserved for production training, and spot to fault-tolerant batch jobs. Red flag: spot for real-time serving or skipping reserved capacity analysis.
Describe a basic lifecycle policy to manage cloud storage costs
This tests cost optimization via tiered storage and automated expiration. Strong answers list transitions from Standard to IA to Glacier, then deletion after set days, plus retrieval costs. A red flag is using manual scripts instead of native lifecycle rules.
How do you adapt ML training for spot instance interruptions?
Tests resilience under preemption. Strong answers cover frequent checkpoints to durable storage, SIGTERM handling, idempotent retries with budgets, and compute-state separation. Red flag: saving checkpoints only on local ephemeral disks or solely at epoch end.
Three ways to cut LLM inference cost
Quantization to shrink the model, continuous batching with paged attention to raise GPU utilization, and distillation or smaller routed models for easy queries.
Design a near real-time cost visibility system for ML teams
Tests cost attribution across shared ML infrastructure and streaming pipeline design. Strong answers combine billing exports with resource labels, sub-hour aggregation, and anomaly detection for training spikes.
Design a showback or chargeback system for ML infrastructure costs
Tag workloads to cost centers; define shared-resource formulas; automate reconciliation; use showback.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles