Git is fine for my code, but what about my data and my models? I keep losing track of which version of the dataset produced which model results. Is there a tool that links the code, data, and model version together?
DVC is the standard tool for versioning machine learning experiments by linking code in Git with external data storage via lightweight metadata pointers.
2 answers
You should stop trying to track manual files and implement DVC alongside your Git workflow. It uses your existing git infrastructure to version data and models by storing pointers in text files while the actual large binary data resides in cloud storage.
I remember spending two weeks hunting down a ghost model because a colleague swapped the source CSV without updating the deployment metadata, and frankly, that taught me to never trust unmanaged files.
You need to move away from local file tracking and adopt a proper lineage tool like MLflow or DVC. These platforms maintain a strict mapping between the snapshot of your code, the specific data version, and the resulting artifact so that your production pipelines are actually reproducible.
Governance is not a suggestion in enterprise environments, it is a prerequisite for survival.
