Closed API vs open-weight models for production

WHAT THIS TESTS This evaluates whether you reason about a model choice as a system and business decision spanning capability, cost, data governance, latency, and team capacity, rather than picking whichever scores highest on a benchmark.

A GOOD ANSWER COVERS Closed-source APIs typically offer frontier reasoning quality with no infrastructure to manage; you call an endpoint and get results immediately. The costs are recurring per-token billing that can dominate at high volume, vendor lock-in, dependence on the provider's uptime and rate limits, and the fact that your prompts leave your security boundary, which can break compliance for regulated data. Open-weight models like Llama 3 let you self-host, keeping data in your own environment, fine-tune freely, control versions, and convert variable per-token cost into more predictable hardware spend. The price is owning GPU provisioning, model serving, autoscaling, monitoring, and the MLOps expertise to do it reliably. Quality on the very hardest tasks may trail the top closed models. The right answer maps these to the specific application's volume, data sensitivity, latency budget, and team skills.

COMMON WRONG ANSWERS Deciding only on benchmark numbers; ignoring data privacy and compliance; assuming open-source is always cheaper without counting GPU and ops cost; forgetting vendor lock-in and rate limits; overlooking the team's operational maturity.

LIKELY FOLLOW-UPS At what volume does self-hosting beat API cost? How do you handle data residency? What about a hybrid: API for hard queries, open model for the rest? How do you mitigate lock-in?

ONE CONCRETE EXAMPLE A healthcare app processing protected patient data cannot send it to a third-party API, so it self-hosts Llama 3 on-prem despite the ops burden. A low-volume consumer prototype with no sensitive data instead uses a closed API to ship fast and avoid running GPUs.