Overmind: Cut PyTorch model load time 15s to 0.2s with a zero-copy shared cache (github.com) 1 points by westurner 13d ago ↗ HN
[–] westurner 13d ago ↗ meshy-dev/overmind: Cut PyTorch model loading time from 15s to 0.2s with zero-copy shared memory caching.> Fast Model loading: First load caches to shared memory; subsequent loads are ~5x faster> Process-agnostic - Cache persists across process restarts via a background server> Non-intrusive - Just add one line of code; no changes to model loading logic
1 comment
[ 3.6 ms ] story [ 9.8 ms ] thread> Fast Model loading: First load caches to shared memory; subsequent loads are ~5x faster
> Process-agnostic - Cache persists across process restarts via a background server
> Non-intrusive - Just add one line of code; no changes to model loading logic