Design an MLOps platform for a mid-sized company: components and build-vs-buy trade-offs

WHAT THIS TESTS: This question evaluates your ability to design a pragmatic MLOps architecture for a mid-sized company with finite engineering resources and to justify build versus buy decisions using total cost of ownership, maintenance burden, and time-to-value rather than technology preference alone. Interviewers want to see that you understand which layers are commodity infrastructure and which are competitive differentiators, and that you can sequence investments to avoid building a complex platform before the basics are solid.

A GOOD ANSWER COVERS: A strong response names six foundational components in priority order. First, the data estate including raw storage, governance policies, and lineage tracking because garbage data destroys model performance regardless of algorithm quality. Second, feature pipelines and a feature store to enable reuse and consistency between training and serving. Third, a model registry with versioned artifacts and metadata so teams can reproduce experiments and rollback safely. Fourth, CI CD and CT pipelines that automate testing, validation gates, and retraining triggers rather than relying on manual notebook execution. Fifth, monitoring and observability covering data drift, concept drift, and latency SLAs. Sixth, the serving layer for online and batch inference, which should be addressed only after the first five are stable. For build versus buy, the answer should map each layer to a simple rule: buy off the shelf tools for orchestration, experiment tracking, and monitoring because they are solved problems; build custom code only for feature engineering logic and model architectures that directly create business differentiation. The justification must include real constraints such as a mid sized team of five to fifteen engineers cannot afford to maintain a custom Kubernetes scheduler or feature store when managed alternatives exist.

COMMON WRONG ANSWERS: Red flags include proposing a fully custom orchestrator or feature store without quantifying the two to three full time engineers required to maintain it. Another red flag is prioritizing exotic real time serving with custom GPU clusters before basic data governance and monitoring are in place. Candidates who treat build versus buy as a religious debate rather than a total cost of ownership calculation also signal immaturity. Similarly, ignoring governance, access control, or lineage tracking suggests a prototype mindset rather than production engineering.

LIKELY FOLLOW-UPS: Interviewers often push deeper by asking how you would handle a specific latency requirement such as one hundred millisecond p99 inference, how you would design retraining triggers based on data drift thresholds, or how you would justify the operational cost of a feature store to a skeptical CFO. They may also ask which specific tools you would evaluate first, such as Kubeflow, MLflow, Tecton, or SageMaker, and how you would migrate from a notebook based workflow without halting existing model deliveries.

ONE CONCRETE EXAMPLE: Imagine a mid sized e commerce company with eight data scientists and three platform engineers. Instead of building a custom Airflow competitor, you purchase a managed workflow service and invest custom engineering hours into a domain specific feature store schema for user session embeddings because that directly improves recommendation relevance. You defer real time serving from a custom Flask cluster to a managed autoscaling endpoint until monitoring proves that batch inference is insufficient. This sequence keeps the team focused on revenue generating model improvements rather than infrastructure maintenance.

Source: MLOps Architecture: End-to-End Design for Production-Grade ML and LLM Systems - DEV Community