Model 4 bit inference 4.2x faster than 16 bit with full HF support (twitter.com) 2 points by convexstrictly 3y ago ↗ HN
[–] convexstrictly 3y ago ↗ Release notes: https://github.com/TimDettmers/bitsandbytes/releases/tag/0.4...
1 comment
[ 3.2 ms ] story [ 19.1 ms ] thread