[–] yoss90 1y ago ↗ JPEG is unlikely to do a good job in compressing LLM weights, but here is different approach from this weekend (with benchmarks and code).https://github.com/nadavrot/legday/#how-it-works
[–] billconan 1y ago ↗ does this make sense?JPEG compression is based on the fact that natural images are of low frequency. Are weights also low frequency?
2 comments
[ 1.3 ms ] story [ 21.4 ms ] threadhttps://github.com/nadavrot/legday/#how-it-works
JPEG compression is based on the fact that natural images are of low frequency. Are weights also low frequency?