Q8 KV cache lets a 30B model fit 100K context on a 24 GB RTX 5090 (buraak.com) 3 points by bozdemir 2mo ago ↗ HN
0 comments
[ 4.1 ms ] story [ 11.2 ms ] threadNo comments yet.