DVC

Full Description

DVC (Data Version Control) is an open-source tool that brings Git-like practices to machine learning projects. It lets teams version datasets, models, and other large files without storing them in Git, using lightweight metafiles and a content-addressed cache. DVC supports multiple remote backends (e.g., S3, GCS, Azure, SSH) for pushing and pulling data, enabling efficient collaboration and storage deduplication. Beyond data versioning, DVC provides reproducible pipelines defined in dvc.yaml, capturing stages, dependencies, and outputs so experiments can be rerun consistently (e.g., with dvc repro). It includes experiment tracking (dvc exp), parameters management (params.yaml), and metrics/plots to compare runs. DVC integrates smoothly with any language or framework, works alongside Git and CI/CD, and helps establish clear lineage and provenance for datasets and models—making ML workflows more reliable, collaborative, and scalable.