Design a multi-tenant ML platform with isolation, security, and cost attribution
Tests mapping tenancy to compute, network, and identity primitives. Strong answers compare hard vs soft isolation, use namespaces or node pools with network policies and IAM, and enforce chargeback via resource quotas and labels.
What's really being asked
This question evaluates whether you can translate abstract business requirements like tenant isolation, security, and cost accountability into concrete infrastructure primitives. At the senior level, interviewers care about trade-offs between hard and soft multitenancy, blast radius containment, and operational cost rather than just naming technologies.
The full answer
First, pick a tenancy model. Hard isolation uses dedicated clusters, node pools, or subscriptions per team for the strongest boundary but higher cost. Soft isolation shares underlying compute and uses logical separation for efficiency. Second, enforce resource isolation. In Kubernetes this means namespaces with ResourceQuotas and LimitRanges; in VMs this means separate VM scale sets or dedicated hosts with CPU and memory constraints. Third, establish security boundaries. Use network policies or NSGs to block lateral movement, enforce pod security standards or sandboxed runtimes, and scope IAM so tenant admins cannot affect other tenants. Fourth, implement cost attribution. Label every resource with tenant and project tags, aggregate metrics through a metering pipeline, and expose chargeback dashboards or quota enforcement to drive accountability.
The mistakes people make
Proposing a single shared cluster with only RBAC for separation. This ignores noisy neighbor problems and lateral network movement. Suggesting physical hardware per tenant without discussing cost or utilization trade-offs. Conflating user identity with tenant identity, which breaks when teams have service accounts or external collaborators. Ignoring data plane isolation for shared storage or feature stores, which is critical in ML where datasets and model artifacts cross boundaries.
What usually comes next
How would you handle a tenant that needs GPUs while others run CPU-only workloads? What happens when one tenant submits a training job that exhausts cluster autoscaling limits? How do you isolate shared ML metadata stores or model registries? How do you onboard a new tenant without redeploying the platform?
A concrete example
An enterprise runs a central Kubernetes cluster for five data science teams. Each team gets a namespace with a ResourceQuota of 100 CPUs and 500 GB memory. A cluster autoscaler manages two node groups: CPU-only and GPU-enabled. Network policies block all cross-namespace traffic except to shared services like the central model registry, which authenticates by tenant identity. Cost allocation uses namespace labels scraped by a metering tool and fed into an internal billing API. When Team A submits a large distributed training job, it cannot evict Team B's inference pods because priority classes and preemption policies are scoped per namespace.
Interview question
When building a soft-multi-tenant Kubernetes ML platform, which combination best prevents noisy neighbors, blocks lateral movement, and enables accurate chargeback without dedicated infrastructure per tenant?
- a.Dedicated node pools per tenant, separate cloud subscriptions, and pod security standards
- b.Namespaces with ResourceQuotas and LimitRanges, network policies blocking cross-namespace traffic, and tenant labels feeding a metering pipelineCorrect
- c.Namespace RBAC, resource requests on pods, and user identity labels for cost tracking
- d.A single shared namespace with RBAC, pod security standards, and network policies allowing cross-tenant traffic to shared services
Why? this is the answer
B correctly combines logical resource boundaries, network segmentation, and labeled metering for cost attribution. A tempts by mentioning RBAC and labels but fails to enforce hard resource limits and conflates user identity with tenant identity; C proposes hard isolation that violates the soft-tenancy constraint; D relies on RBAC-only separation in shared namespaces and permits lateral movement.
Just read this? Test yourself on what you have been reading.
Read the original → learn.microsoft.com
- #mlops
- #multitenancy
- #kubernetes
- #architecture
- #cost-optimization
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on mlops — each one lists the topics its interview covers.
See open roles