More in MLOps & Infrastructure — page 4
How do you give read-only access to a shared cloud storage bucket?
WHAT IT TESTS: Least-privilege IAM for shared data pipelines. ANSWER OUTLINE: Bind an IAM role with read permissions to the team at the bucket level, avoid object-level ACLs, and mount read-only on training VMs.
Propose an architectural solution for contended GPU training resources
Tests multi-tenant GPU scheduling design at scale. Great answers tier jobs by checkpointability, apply quota-based preemption, mix spot and on-demand instances, and use MIG or time-slicing to bin-pack. Red flag: buying GPUs without scheduling logic.
Design a multi-tenant ML platform with isolation, security, and cost attribution
Tests mapping tenancy to compute, network, and identity primitives. Strong answers compare hard vs soft isolation, use namespaces or node pools with network policies and IAM, and enforce chargeback via resource quotas and labels.
How would you design a reproducible ML training pipeline?
Tests if you can version ML's three moving parts: code, data, and environment. Good answers cover Git for code, DVC or lakehouse versioning for data, and Docker plus locked dependencies for environments.
What trade-offs decide managed ML platforms versus open-source Kubernetes?
WHAT IT TESTS: Ops overhead vs speed for ML infra. ANSWER OUTLINE: Weigh total cost plus hidden engineering headcount, lock-in vs flexibility, and audit feature gaps. RED FLAG: Recommending open-source purely to cut cost while ignoring the 2-4 person tax.
What is a feature store and how does it prevent training-serving skew?
This tests training-serving consistency via centralized feature management. Covers offline batch storage, online serving, shared transformations, and alternatives like ad-hoc ETL. A red flag is calling it just a database and ignoring point-in-time correctness.
What are the essential components of an end-to-end ML platform?
Tests mapping the ML lifecycle to infrastructure. A strong answer walks through six stages: data ingestion, feature store, training pipeline, model registry, deployment/serving, and monitoring with feedback loops.
How would you architect dynamic training resource provisioning?
WHAT IT TESTS: elastic ML infrastructure and cost control. A GOOD ANSWER: routes jobs to right-sized instances, provisions managed jobs using spot, and auto-tears down on completion. RED FLAG: suggesting always-on clusters or serverless GPU training.
Design concept drift detection with automated retraining safeguards
WHAT IT TESTS: MLOps design separating drift detection, triggers, and stability controls. ANSWER OUTLINE: Baseline monitors raise CloudWatch alarms; EventBridge triggers retraining with cooldowns; model registry gates promotion.
How do you programmatically promote a retrained model to production?
WHAT IT TESTS: Gated promotion balancing statistics and safety. ANSWER OUTLINE: Compare on held-out data using significant metric uplift, schema, latency, and drift checks before shadow release. RED FLAG: Using training accuracy without variance checks.
How would you implement automated data validation before training?
WHAT IT TESTS: Pipeline gatekeeping and failure isolation in production ML. ANSWER OUTLINE: Enforce schema contracts, halt training on failure, quarantine bad batches, and alert owners. RED FLAG: Manual reviews or soft warnings letting bad data into training.
What triggers automatic full retraining in an ML pipeline?
WHAT IT TESTS: Your grasp of data, code, model, and schedule-driven automation in MLOps. ANSWER OUTLINE: A strong answer lists four triggers: fresh data, code changes, model drift, and scheduled cadence.
How do you monitor thousands of per-customer models as a fleet?
Tests fleet-level statistical aggregation versus per-instance alerting. Strong answers propose tiered telemetry, cohort baselining for drift, and hierarchical alerting to prevent fatigue.
Design an automated system to diagnose model performance drop root causes
Tests causal attribution between pipeline bugs and drift. Strong answers sequence schema/null audits, feature drift via PSI/KS, then concept drift via holdout decay. Red flag: skipping pipeline checks to retrain immediately.
Model output distribution shifts. What are root causes and next steps?
This tests covariate vs label shift vs concept drift when outputs shift. A strong answer checks features before labels, then feedback loops or staleness. A red flag is generic drift without separating P(X), P(Y), and P(Y|X).
Design a system to monitor a real-time prediction service for feature drift
WHAT IT TESTS: production ML observability beyond accuracy checks. ANSWER OUTLINE: async feature logging, distribution comparison via PSI/KS against training baseline, and threshold-based anomaly alerts.
Differences between monitoring a traditional REST API and a production ML model
WHAT IT TESTS: Awareness that ML fails via data decay, not code bugs. ANSWER OUTLINE: Contrast latency/errors with ML signals like data drift and training-serving skew against baselines, noting ground truth delays.
Design a multi-model serving architecture for infrequently used models
Tests sparse-traffic cost efficiency via shared compute and dynamic loading. Strong answers: LRU cache on fast storage, scale-to-zero with async preload, pinned hot models, per-model quotas.
Design cost-effective inference for spiky traffic without idle GPUs
Tests designing inference that cuts idle GPU cost during troughs yet handles spiky peaks with low latency via SageMaker blue/green fleets, production variants, and CloudWatch baking periods. Red flag: always-on GPU pools with naive auto-scaling.
How would you design an A/B test for two live ML models?
Tests production experimentation rigor beyond random splitting. Strong answers cover: consistent user hashing for sticky assignment, isolated feature stores, guardrail metrics, and pre-calculated statistical power.