Infrastructure
110 bites tagged Infrastructure — interview questions with model answers, and 60-second explainers.
Describe the difference between online and batch inference.
Online uses autoscaling APIs for millisecond-to-second latency; batch uses scheduled compute for minute-to-hour latency. Your grasp of serving patterns and infra tradeoffs.
How would you systematically debug an inference API latency breach?
This tests structured debugging across the full inference stack. A strong answer traces the request path from ingress to GPU, splits TTFT from token-generation latency, inspects queuing and batching, then applies targeted fixes.
Describe the difference between online and batch inference.
Online serves single requests in ms on live endpoints; batch processes data offline with elastic compute. Latency vs throughput trade-offs in serving. Treating batch as slow online inference or ignoring SLA and cost.
What infrastructure is needed for a Continuous Training pipeline?
Tests event-driven ML system design beyond CI/CD. Strong answers name orchestrators, feature stores, model registries, and validation gates, mapping triggers to retraining and promotion. Red flag: conflating CT with CI/CD or skipping model validation.
Key differences between traditional and ML CI/CD pipelines?
Contrast code deploys with data versioning, model registries, and retraining; note holdout eval. ML CI/CD manages data and model lineage, not just code. Treating the model as a static binary ignoring data or retraining context.
Strategies to reduce a 5GB ML Docker image size
Tests multi-stage build hygiene and ML bloat reduction. Strong answers use multi-stage builds, strip CUDA dev libs, use slim bases, and collapse cache cleanup into one RUN. Red flag: rm -rf in a separate RUN step, which still bloats the layer.
How do you version and distribute Docker dev environments consistently?
Tests immutable dev environment distribution. Strong answers cover: versioned Dockerfiles in Git, immutable image tags pushed to a registry, and enforcing identical pulls for CI and developers. Red flag: using the "latest" tag or local Dockerfile rebuilds.
What problems does a Feature Store solve in ML systems?
Tests understanding of feature store value beyond storage. Great answers cover: feature reuse across teams, managed transformation pipelines, and online/offline consistency to prevent training-serving skew. Red flag: calling it simply a database or cache.
Describe feature store architecture and training-serving skew
This tests FTI pipeline glue and dual-store skew elimination. A strong answer lists offline and online stores, shared transformation logic for consistent compute, and point-in-time correctness. A red flag is calling it merely a database or cache.
How do you guarantee identical feature engineering for training and real-time inference?
Tests unifying feature engineering across batch and online paths to eliminate skew. Answer: shared transformation libraries, versioned feature stores, and logged feature validation. Red flag: separate training and serving code without a single source of truth.
Explain ML pipelines and typical CI/CD/CT components
Tests if you separate code CI/CD from model CT and grasp ML automation. Cover source control, build, tests, deploy for code; data validation, training, evaluation, promotion for CT. Red flag: treating ML like software CI/CD and ignoring data or registry gates.
What are the primary differences between traditional DevOps and MLOps?
Tests if you know MLOps extends CI/CD to data and probabilistic assets. Contrast code artifacts with models and datasets. Distinguish commit triggers from drift or retraining triggers. Red flag: treating models as static binaries and ignoring data lineage.
Centralized vs Decentralized ML Platforms
A centralized ML platform trades team autonomy for standardization, while decentralized platforms embed ML tooling inside product teams. Centralized suites drown in ticket queues; decentralized ones duplicate cost and security holes without strong governance.
Right-Size Inference and Stop Paying for Idle GPUs
Instance right-sizing matches inference to the smallest hardware that serves it without choking. It matters when GPU endpoints idle at 10% utilization. The footgun is copying your training spec into production; inference rarely needs that memory or multi-GPU.
Docker Image vs. Container: Blueprint vs. Runtime
A Docker image is a read-only blueprint; a container is a live instance with a writable layer. You build an image once in CI and run many containers from it in production. The footgun is mutating a running container without updating the image recipe.
Model Server: The MLOps Deployment Bridge
A model server bridges ML training and production, operationalizing models within your release cycle. Use it when models must become first-class CI/CD citizens. The footgun is treating deployment as a one-time handoff rather than repeatable infrastructure.
Shadow Deployment: Test Models on Real Traffic
Shadow deployment runs a new model on real traffic without serving its predictions, letting you catch data drift before users are affected. It is the safest production validation method, but teams often forget to monitor its latency and resource costs.
How do you mitigate peeking in experiment infrastructure?
Lock results behind minimum samples; auto-correct via sequential testing; hide early metrics and require stop approval. infrastructure enforcing valid reads through gates and incentives. relying on training not systems.
Should we build or buy an A/B testing platform?
This tests build-vs-buy judgment for experimentation infrastructure. Strong answers cover build for warehouse metrics and cache control; buy for proven stats and front-end speed. Red flag: answering with cost alone or assuming in-house is always superior.
How do Docker images and containers differ and relate?
This tests your grasp of the immutable template versus mutable runtime boundary. A good answer: an image is a read-only layered template with code and dependencies; a container is a runnable instance with a writable layer on top.
When is streaming better than batch, and what are its infrastructure challenges?
Tests if you separate low-latency decisions from analytics and know ops pain. Good answer: fraud detection vs batch; cite exactly-once, state recovery, backpressure, schema drift. Red flag: calling streaming faster batch while ignoring backpressure or state.
Explain SPF, DKIM, and DMARC roles and implementation tasks
SPF authorizes IPs, DKIM signs content, DMARC aligns both to the From domain and enforces policy plus reporting. Your grasp of layered email auth and operational deployment.
What is an artifact repository and why not just a shared filesystem?
Tests your grasp of immutability, metadata, and access control for build artifacts. A strong answer covers versioning, checksums, RBAC, and API retrieval that NFS lacks. Red flag: saying a shared filesystem is simpler and therefore enough for production.
How would you implement zero-downtime secrets rotation?
Inventory secrets and app caching; baseline monitoring; dual-phase rotation with overlapping secrets; verify before revoking old. rotating credentials without downtime.
Get Infrastructure bites daily.
Five a day, five minutes, offline. With quizzes so it sticks.
Open testing — you’ll join as an early tester.