Intermediate everything in MLOps & Infrastructure
Epsilon in differential privacy and its trade-offs
Epsilon is the privacy budget bounding how much one record can change outputs; smaller epsilon means stronger privacy but more noise and lower accuracy.
Declarative vs imperative ML platform design
Declarative GitOps gives auditable, reproducible, reviewable desired-state config with strong governance but a steeper learning curve; imperative SDKs are flexible and fast for scientists but harder to…
Flask/Gunicorn vs Triton/TorchServe for serving
Flask is simple and flexible but lacks dynamic batching, GPU scheduling, and multi-model management; Triton/TorchServe add those plus metrics and versioning.
Hybrid parallelism for large-model training
Split the model itself via tensor or pipeline parallelism so each replica is smaller, shrinking gradient all-reduce; combine with data parallelism in 2D/3D.
Why design ML pipeline steps to be idempotent?
Re-running a step with the same input yields the same result and no duplicate side effects; enables safe retries and backfills.
Concept drift vs data drift in production models
Data drift is a shift in input distribution P(X); concept drift is a shift in the relationship P(Y|X).
What is a model registry and how does it enable CD?
A registry versions models with metadata, lineage, and stage tags; CD watches stage transitions to trigger deploys.
What is shadow deployment for ML models?
New model receives mirrored live traffic but its predictions are logged, not served; validates real-world behavior and latency without user risk.
Three ways to cut LLM inference cost
Quantization to shrink the model, continuous batching with paged attention to raise GPU utilization, and distillation or smaller routed models for easy queries.
Serving for online and batch predictions
A low-latency online path (synchronous, autoscaled, real-time features) and a high-throughput batch path (parallel, cost-optimized, large jobs) sharing one model artifact and feature definitions.
Diagnosing poor distributed training scaling
Communication overhead (gradient all-reduce, interconnect), data-loading starvation, load imbalance, and small per-GPU batches; profile with the PyTorch profiler and Nsight.
Sub-20ms online feature serving
An in-memory key-value store (Redis) as the online feature store, precomputed features, streaming updates, and offline-online consistency.
Versioning a 10TB dataset as code
Treat data like code via content-addressed pointers in Git while bytes live in object storage; dedupe by hashing so versions share unchanged files.
Directed Acyclic Graph (DAG) for Workflows
A DAG models a workflow as tasks (nodes) connected by dependency edges with no cycles, so a scheduler knows the valid execution order. It enables parallelism, safe retries, and idempotent reruns, and underpins orchestrators like Airflow for ML pipelines.
Staging Environments for ML Pipelines
A staging environment mirrors production so models and pipelines are validated on production-like data and infrastructure before release. It catches drift, integration breaks, and serving regressions early, making promotion to production a safe, repeatable…
Most LLM Apps Need Workflows Not Agent Frameworks
Most LLM apps ship faster and more reliably as deterministic workflows than autonomous agents. Plain Python with structured outputs and local functions beats CrewAI and LangGraph for debugging. Map control flow in code before importing any agent framework.
ORPilot JSON IR Ends Solver Lock-In
ORPilot's open-source IR captures optimization models as solver-agnostic JSON, letting teams swap solvers or update data without calling the LLM again. It separates model structure from solver syntax, making LLM-generated OR models reproducible in production.
Default Churn Thresholds Waste $86 per Customer
90% of 36 IBM Telco churn analyses use F1 and a 0.5 threshold, assuming equal costs for false positives and negatives. That is wrong by 13x, burning $86 per customer, or $8.6M at 100k subscribers. Swap accuracy for profit curves tied to LTV and CAC.
Design a cost-aware ML training platform for heterogeneous hardware
Tests hardware abstraction and cost-aware cross-accelerator scheduling. Strong answers cover a device-agnostic spec, a performance predictor, a cost-per-step model, and bin-packing against spot prices. Red flag: ignoring per-step cost and migration overhead.
How does a model registry differ from cloud storage like S3?
This tests model governance beyond raw storage. A strong answer contrasts storage with stage transitions, lineage, and ACLs, then lists metadata like metrics, dependencies, and schemas. A red flag is treating S3 folders with naming conventions as a registry.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles