How do you ensure ML experiment reproducibility beyond random seeds?

WHAT THIS TESTS: This question tests whether you view reproducibility as a socio-technical system or merely a code setting. Interviewers want to see if you understand that ML experiments depend on four mutable pillars: data, code, environment, and compute configuration. Senior candidates should demonstrate experience building guardrails for teams, not just personal best practices, and show awareness that reproducibility failures usually stem from invisible state changes rather than explicit bugs.

A GOOD ANSWER COVERS: First, data versioning and lineage. You should mention tools like DVC that version datasets as metafiles in Git, ensuring every experiment points to an immutable data snapshot rather than a mutable file path. Second, environment capture. Strong candidates cite container images with locked digests, conda lockfiles, or poetry export to eliminate dependency drift. Third, pipeline codification. Describe defining training steps in declarative configuration files or metafiles that Git can diff, making the execution graph reviewable and rerunnable by any team member. Fourth, experiment tracking and artifact management. Mention logging hyperparameters, metrics, code commit hashes, and model artifacts to a central store so results are queryable and comparable. Fifth, infrastructure determinism. Briefly note fixed compute types, CUDA versions, and deterministic GPU operations to isolate hardware variance.

COMMON WRONG ANSWERS: The biggest red flag is stopping at random seeds and Python hash randomization. Another weak pattern is suggesting manual documentation or shared folders without versioning. Some candidates over-engineer by proposing heavy proprietary platforms without mentioning Git-based workflows or lightweight metafile approaches that scale from laptop to CI cluster.

LIKELY FOLLOW-UPS: How do you reproduce an experiment from six months ago when the original data has been overwritten? How do you handle non-deterministic GPU operations or data augmentation pipelines? What is your strategy when dataset size makes full Git versioning impractical? How do you enforce these practices across a team of ten researchers without creating friction?

ONE CONCRETE EXAMPLE: On a computer vision project, we used DVC to version multi-terabyte image directories stored in S3, committing only .dvc and dvc.yaml files to Git. Every training run was executed through DVC pipelines so that reproducing an experiment required only git checkout and dvc repro. We paired this with Docker images pinned by digest and MLflow for metric logging. When a teammate questioned a sudden accuracy drop, we checked out their exact commit, DVC pulled the corresponding data snapshot, and reproduced the identical model weights within an hour.

Source: doc.dvc.org