Everything in MLOps & Infrastructure, page 7
How to establish data lineage and reproducibility for hundreds of ML models
This tests MLOps traceability architecture. A strong answer proposes a unified metadata graph linking raw data, feature transforms, dataset versions, training runs, and deployed models via automated hooks.
Design a sub-50ms real-time bidding feature pipeline
Tests merging batch historical and streaming data under sub-50ms latency. Strong answers use dual paths: batch backfills a KV store, streaming writes to an in-memory cache, serving merges both at request time. Red flag: one database without hot-cold split.
How would you design automatic data drift detection for production inference?
This tests reference-vs-live monitoring architecture. A strong answer chunks data, runs univariate per-feature drift, adds multivariate PCA or domain classifiers, and ranks threshold alerts. A red flag is checking aggregate metrics instead of feature shifts.
How would you scale 1TB Pandas feature computation across machines?
This tests memory limits and distributed migration. A strong answer contrasts single-machine tactics, column pruning and efficient dtypes, with distributed frameworks like Dask or Spark, noting shuffle costs and API parity.
Describe feature store architecture and training-serving skew
This tests FTI pipeline glue and dual-store skew elimination. A strong answer lists offline and online stores, shared transformation logic for consistent compute, and point-in-time correctness. A red flag is calling it merely a database or cache.
What data validation strategy prevents new categories from breaking your encoder?
Lock categorical domains in a schema, reject unseen categories before encoding, and use an OOV bucket as fallback.
ETL vs ELT: when to prefer each for ML?
Tests architecture tradeoffs. Contrast ETL's pre-load transformation with ELT's in-warehouse transformation, map structured legacy to ETL and raw lakes to ELT, and cite ML iteration needs. Red flag: calling one better ignoring compute placement or volume.
Design ingestion for clickstream and batch product metadata
Use a data lake for raw data, a feature store for joins, columnar formats for batch, and row formats for events.
How do you guarantee identical feature engineering for training and real-time inference?
Tests unifying feature engineering across batch and online paths to eliminate skew. Answer: shared transformation libraries, versioned feature stores, and logged feature validation. Red flag: separate training and serving code without a single source of truth.
Design a robust automated testing strategy for ML models before production
Statistical offline thresholds, shadow-canary launches, input drift detection, and rollbacks tied to KPIs.
Design an MLOps platform for a mid-sized company: components and build-vs-buy trade-offs
Tests pragmatic scoping and build-vs-buy reasoning. Strong answers rank data estate, feature store, registry, CI/CD/CT, and monitoring above exotic serving, buying commodity and building differentiators. Red flag: custom orchestrators or missing governance.
What production metrics and auto-thresholds trigger model retraining?
This tests production monitoring maturity. A strong answer covers technical drift metrics, business KPIs tied to model decisions, and automated thresholds that page or trigger CI/CD retraining.
Explain ML pipelines and typical CI/CD/CT components
Tests if you separate code CI/CD from model CT and grasp ML automation. Cover source control, build, tests, deploy for code; data validation, training, evaluation, promotion for CT. Red flag: treating ML like software CI/CD and ignoring data or registry gates.
Why version code, data, and models in MLOps?
Tests immutable lineage across code, data, and models. Strong answers cover content-addressed data, git commits, a model registry linking both, and CI triggers on any change. Red flag: saying git alone handles data and models.
What are the primary differences between traditional DevOps and MLOps?
Tests if you know MLOps extends CI/CD to data and probabilistic assets. Contrast code artifacts with models and datasets. Distinguish commit triggers from drift or retraining triggers. Red flag: treating models as static binaries and ignoring data lineage.
Describe the key stages of a typical ML lifecycle
It tests end-to-end systems thinking beyond notebook prototyping. Strong answers list: problem framing, data processing, model development, deployment, and monitoring with retraining. A red flag is skipping data validation or post-deployment observability.
Model Risk Management: The Immune System for Production Models
Model Risk Management treats every deployed model as a liability that can silently decay. Banks use it to stop bad predictions from becoming bad decisions. The footgun is treating validation as a one-time checkbox instead of continuous governance.
Centralized vs Decentralized ML Platforms
A centralized ML platform trades team autonomy for standardization, while decentralized platforms embed ML tooling inside product teams. Centralized suites drown in ticket queues; decentralized ones duplicate cost and security holes without strong governance.
Dynamic Fan-out/Fan-in Pipelines
Dynamic fan-out/fan-in spawns parallel tasks from runtime data, then gathers results. Use it when input counts vary, like processing a daily changing set of files. The footgun is a fan-in task that hangs waiting for branches lost to partial failure.
Parameterization: One Pipeline, Any Environment
Externalize every path, hyperparameter, and compute setting so one pipeline runs unchanged across dev, staging, and production. This enables reproducible experiments and safe CI/CD. The footgun is branch-per-environment repos that silently diverge.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles