Advanced interview questions in MLOps & Infrastructure, page 2
Design a multi-model serving architecture for infrequently used models
Tests sparse-traffic cost efficiency via shared compute and dynamic loading. Strong answers: LRU cache on fast storage, scale-to-zero with async preload, pinned hot models, per-model quotas.
Design an automated system to diagnose model performance drop root causes
Tests causal attribution between pipeline bugs and drift. Strong answers sequence schema/null audits, feature drift via PSI/KS, then concept drift via holdout decay. Red flag: skipping pipeline checks to retrain immediately.
How do you monitor thousands of per-customer models as a fleet?
Tests fleet-level statistical aggregation versus per-instance alerting. Strong answers propose tiered telemetry, cohort baselining for drift, and hierarchical alerting to prevent fatigue.
Design concept drift detection with automated retraining safeguards
Baseline monitors raise CloudWatch alarms; EventBridge triggers retraining with cooldowns; model registry gates promotion.
How would you architect dynamic training resource provisioning?
Routes jobs to right-sized instances, provisions managed jobs using spot, and auto-tears down on completion.
Automated rollback for a failed model deploy
Detect failure via health and metric checks, automatically route traffic back to the last known-good version (blue-green or canary), keep the registry entry but unpromote, and alert.
Design a multi-tenant ML platform with isolation, security, and cost attribution
Tests mapping tenancy to compute, network, and identity primitives. Strong answers compare hard vs soft isolation, use namespaces or node pools with network policies and IAM, and enforce chargeback via resource quotas and labels.
Propose an architectural solution for contended GPU training resources
Tests multi-tenant GPU scheduling design at scale. Great answers tier jobs by checkpointability, apply quota-based preemption, mix spot and on-demand instances, and use MIG or time-slicing to bin-pack. Red flag: buying GPUs without scheduling logic.
Design a cryptographically verifiable ML audit trail from dataset to deployment
Tests cryptographic provenance and tamper-evident ML pipelines. Strong answers cover content-addressed datasets, signed training logs linking code and hyperparameters to model hashes, and deployment signature checks.
Design a defense-in-depth strategy against adversarial evasion on a deployed image classifier
Proactive: adversarial training, preprocessing, ensembles.
Design a showback or chargeback system for ML infrastructure costs
Tag workloads to cost centers; define shared-resource formulas; automate reconciliation; use showback.
Describe an ML workflow with massive egress fees and re-architecture to mitigate
Tests whether you recognize egress spikes when storage and compute cross cloud or region boundaries. Great answers sketch a multi-cloud training pipeline, cite per-GB rates, and propose caching or compute placement. Red flag: suggesting compression alone.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles