Advanced everything in MLOps & Infrastructure, page 2
Architectural challenges for deploying ML models on resource-constrained edge devices
Tests Edge MLOps architecture under severe constraints. Strong answers hit quantization and delta OTA updates for flaky networks, power-aware scheduling, and closed-loop drift detection.
How would you systematically debug an inference API latency breach?
This tests structured debugging across the full inference stack. A strong answer traces the request path from ingress to GPU, splits TTFT from token-generation latency, inspects queuing and batching, then applies targeted fixes.
How would you design safe, automatic schema evolution in CI?
Tests whether you separate schema evolution from semantic validation. Strong answer: versioned data contracts allowing additive enums, unknown-category model buckets, and automated contract negotiation. Red flag: manual allow-lists or disabling validation.
Design multi-tenant GPU cluster scheduling and preemption policies
Tests ability to design fair GPU scheduling preventing starvation and noisy-neighbor issues. Answer: Kueue for fair-share, namespace quotas with MIG, priority classes with backoff.
Design on-demand containerized dev environments for data scientists
Tests multi-tenant notebook infrastructure design. Cover a Notebook Controller, curated Jupyter and VS Code images, namespace isolation with RBAC, resource quotas, and persistent storage. Red flag: a single shared VM without tenancy or idle shutdown.
Strategies to reduce a 5GB ML Docker image size
Tests multi-stage build hygiene and ML bloat reduction. Strong answers use multi-stage builds, strip CUDA dev libs, use slim bases, and collapse cache cleanup into one RUN. Red flag: rm -rf in a separate RUN step, which still bloats the layer.
Describe two secure methods for providing secrets to a running container
Mention runtime mounts like Docker secrets, orchestrator secret injection, and cloud IAM patterns.
How would you design a system to detect training-serving skew using model registry metadata?
This tests statistical monitoring between production data and registry training baselines. Strong answers: schema-bound metadata, incremental stats, drift metrics PSI, tiered alerting. Red flag: schema validation mistaken for drift or manual checks only.
Argue for declarative or imperative feature platforms with trade-offs
This tests whether you weigh control flow against data flow. A strong answer argues from org maturity: declarative systems abstract DAG topology, while imperative ones offer Spark control at the cost of manual idempotency. Red flag: ignoring org culture.
Design system ensuring point-in-time correctness for training data joins
Tests temporal join design to prevent data leakage from slowly changing dimensions. Strong answers use an AS OF join on entity ID and timestamp, materialize features as of label time, and handle late arrivals. Joining on user_id alone is a red flag.
How to establish data lineage and reproducibility for hundreds of ML models
This tests MLOps traceability architecture. A strong answer proposes a unified metadata graph linking raw data, feature transforms, dataset versions, training runs, and deployed models via automated hooks.
Design a sub-50ms real-time bidding feature pipeline
Tests merging batch historical and streaming data under sub-50ms latency. Strong answers use dual paths: batch backfills a KV store, streaming writes to an in-memory cache, serving merges both at request time. Red flag: one database without hot-cold split.
How do you guarantee identical feature engineering for training and real-time inference?
Tests unifying feature engineering across batch and online paths to eliminate skew. Answer: shared transformation libraries, versioned feature stores, and logged feature validation. Red flag: separate training and serving code without a single source of truth.
Design a robust automated testing strategy for ML models before production
Statistical offline thresholds, shadow-canary launches, input drift detection, and rollbacks tied to KPIs.
Design an MLOps platform for a mid-sized company: components and build-vs-buy trade-offs
Tests pragmatic scoping and build-vs-buy reasoning. Strong answers rank data estate, feature store, registry, CI/CD/CT, and monitoring above exotic serving, buying commodity and building differentiators. Red flag: custom orchestrators or missing governance.
Slash Your Cloud Bill by Taming Data Egress Costs
Data egress is the tax for moving data out of a cloud provider's network, a common cost in MLOps when moving models or datasets. To save money, keep compute and data in the same region. The footgun is forgetting that traffic between regions also counts.
Cloud Cost Anomaly Detection: Finding Waste
Think of it as a smoke detector for your cloud bill, distinguishing 'good' growth-related costs from 'bad' waste. It automatically flags unexpected spending spikes from misconfigurations or bugs.
Reserved Instances vs. Savings Plans: Pre-pay for Cloud Discounts
Think of Reserved Instances and Savings Plans as buying cloud compute in bulk for a discount. You commit to a certain usage level for 1-3 years to save money on steady-state workloads like production databases.
Counterfactual Explanations: How to Change a Model's Mind
A counterfactual explanation finds the smallest input change that flips a model's prediction. It's used to give actionable feedback, like telling a user what to change to get a loan approved.
Adversarial Attacks: Fooling Smart Models with Tiny Changes
Adversarial attacks trick ML models with tiny, imperceptible input changes, causing misclassification. It's like a visual illusion for an AI, turning a 'stop sign' into a 'speed limit' sign by altering a few pixels.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles