Q8 KV cache lets a 30B model fit 100K context on a 24 GB RTX 5090 (buraak.com) 3 points by bozdemir 4mo ago ↗ HN
0 comments
[ 3.0 ms ] story [ 14.5 ms ] threadNo comments yet.