Design a showback or chargeback system for ML infrastructure costs

What's really being asked

This question tests whether you can operate at the intersection of infrastructure engineering, data science workflows, and corporate finance. A senior MLOps leader must translate technical resource consumption into accounting-grade cost attribution. The interviewer cares about your grasp of shared-resource allocation, your ability to collaborate with Finance on reconciliation and general ledger mapping, and your change-management judgment about when to use showback versus chargeback.

The full answer

Four things in order. First, identity and tagging: require every training job, inference endpoint, and notebook to carry metadata linking it to a business unit, cost center, and general ledger code, enforced at the orchestrator or API gateway level. Second, shared resource allocation: define formulas for costs that cannot be tagged directly, such as splitting a multi-tenant GPU cluster by fractional utilization or time-sliced occupancy, and doing the same for shared feature stores and object storage. Third, reconciliation pipeline: build an automated process that ingests cloud vendor invoices and Kubernetes metrics, normalizes them into the format Finance requires, and runs on a schedule that meets month-end close deadlines, because invoices often arrive three to twelve days after month end while usage data is continuous. Fourth, organizational rollout: start with showback to expose costs and build trust, then transition to chargeback only after allocation formulas are audited and accepted by budget owners.

The mistakes people make

Three red flags appear often. One, proposing to use raw cloud provider bills as the attribution source without mapping line items to internal cost centers or accounting codes. Two, ignoring shared infrastructure entirely and attempting perfect per-pod billing, which creates a data engineering quagmire and ignores overhead. Three, skipping the finance partnership and designing the system solely around engineering metrics like CPU seconds rather than invoiced dollars, which breaks reconciliation.

What usually comes next

Expect the interviewer to ask how you would handle spot instance preemptions that cause training restarts and blur cost attribution across billing periods. They may also ask what you do when a business unit disputes a chargeback for shared GPU time, or how you would adjust the model when finance changes the chart of accounts.

A concrete example

Imagine a computer vision team running distributed training on a shared eight-GPU node alongside a natural language processing team. The node costs four thousand dollars for the month. Your platform records GPU utilization metrics every minute. At month end, the FinOps pipeline sums each teams fractional GPU-minutes, allocates the four thousand dollars proportionally, appends the relevant GL codes, and delivers a CSV to the ERP system two days before the accounting close. During the first quarter, the platform runs this as showback only; in quarter two, after two dispute cycles and formula sign-off from both team leads and the CFO office, the same feed becomes a formal chargeback.