Skip to content
tezvyn:

Why version code, data, and models in MLOps?

Source: ml-ops.orgMediumHow cards are made

Why version code, data, and models in MLOps?

Tests immutable lineage across code, data, and models. Strong answers cover content-addressed data, git commits, a model registry linking both, and CI triggers on any change. Red flag: saying git alone handles data and models.

What's really being asked

Whether you recognize that machine learning systems are functions of code, data, and compute, so reproducibility requires versioning all three as a unified lineage graph rather than treating ML like traditional software where Git alone is sufficient.

The full answer

First, the why. Training code without the exact data snapshot and model weights is useless for debugging drift or compliance, and data changes are often the biggest source of model behavior change in production. Second, the implementation architecture. Use content-addressed storage for data, such as DVC, lakeFS, or cloud object storage with versioned buckets and cryptographic hashes, so datasets are immutable and referenceable. Use Git for code to capture training scripts, feature engineering logic, and serving code. Use a model registry, such as MLflow Model Registry or a custom metadata store, where every registered model version stores a manifest containing the data version hash, the Git commit SHA, hyperparameters, and dependency lock files. Third, the interconnection mechanism. The manifest acts as the single source of truth that binds the three artifacts together, enabling you to recreate the exact training run. Fourth, automation. CI/CD triggers retraining or redeployment when any of the three assets change, ensuring the lineage stays current without manual bookkeeping. Fifth, environment reproducibility via container images or dependency locks, because code and data still fail if package versions differ.

The mistakes people make

Proposing Git alone for data and models, which breaks down beyond tens of megabytes and lacks content-addressing for datasets. Treating model versioning as sequential file names like model_v1.pkl without metadata linking back to training context. Ignoring data versioning entirely and only tracking code and model checkpoints. Suggesting manual spreadsheets to track which data was used with which model.

What usually comes next

How would you handle terabyte-scale dataset versioning without copying data? How do you reproduce a model when the original training environment no longer exists? How do you version features and feature transformations alongside raw data? What is your rollback strategy when a deployed model was trained on data that is later found to be biased or corrupted?

A concrete example

A fraud detection model starts degrading. You pull the production model version from the registry, read its manifest, check out the exact Git commit for the training code, mount the exact hashed dataset from your content-addressed store, and rerun training in a container built from the locked dependency file. The reproduced metrics match production logs, proving the issue is data drift rather than a code bug.

Interview question

A deployed fraud model degrades. Using immutable lineage best practices, what is the most reliable way to isolate data drift from a code bug?

  • a.Reproduce the exact training run by combining the manifest's commit SHA, dataset hash, and locked dependencies, then verify the metrics match production logsCorrect
  • b.Mount the exact hashed dataset and rerun training using the latest code on the main branch to verify reproducibility
  • c.Retrain the model on the current production dataset using the original hyperparameters; if performance improves, the cause is data drift
  • d.Compare the current production Git branch against the commit SHA stored in the model manifest to spot code changes
Why?

Reproducing the full training context from the manifest proves the model still yields the same metrics, confirming that production degradation is due to data drift rather than a code bug. Option B is tempting because it uses the exact dataset, but swapping in the latest code introduces a new variable and breaks the lineage chain needed for a valid comparison.

Just read this? Test yourself on what you have been reading.

Read the original → ml-ops.org

You just looked this up. Could you explain it out loud?

That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.

The iPhone app is on the way

We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.

Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.

Get it on Google PlayiPhone app coming soon

We are hiring for this. Open roles that interview on mlops — each one lists the topics its interview covers.

See open roles