Xvc: A CLI tool to manage data and ML pipelines in Rust (GPL3) (github.com) 2 points by emresahin 1y ago ↗ HN
[–] emresahin 1y ago ↗ Xvc is an alternative to Git-LFS, git-annex (in data management) and DVC (in data and pipelines management.)- Track binary/unstructured data and artifacts along with the code- Run pipelines composed of steps depending on files, globs, URLs, SQLite queries, regexes, hyperparameter files etc. (in parallel)- Store binary data files with local, rsync and S3 compatible storages, separately from Git repository- Automates git commands to commit tracked metadata files but can also work without git
1 comment
[ 0.15 ms ] story [ 9.2 ms ] thread- Track binary/unstructured data and artifacts along with the code
- Run pipelines composed of steps depending on files, globs, URLs, SQLite queries, regexes, hyperparameter files etc. (in parallel)
- Store binary data files with local, rsync and S3 compatible storages, separately from Git repository
- Automates git commands to commit tracked metadata files but can also work without git