DeltaTensors: Git for model fine-tunes that also saves you storage

Imagine Git for model fine-tunes that also saves you storage.

DeltaTensors compresses fine-tuned model checkpoints by storing the weight difference from the base model instead of another full copy of the model.

If you have a bunch of fine-tunes or checkpoints from the same base model, you keep the base once and store a much smaller `.wdelta` for each version.

On a Qwen2.5-0.5B fine-tune, a 953 MB model compressed to a 294 MB delta. Reconstructing it changed perplexity from 19.11 to 19.22.

It works after training, so you do not need to use LoRA or change how you fine-tune the model. You can take an existing full fine-tune and compress it afterward.

It also supports chained deltas for model version history, Hugging Face Trainer checkpoints, and streaming compression/reconstruction so you do not need to load two full models into memory.

https://github.com/AaravGaurdev/deltatensors