Speculative KV coding: ~4× losslessly compressed KV cache using a small model (fergusfinn.com) 2 points by somnial 4mo ago ↗ HN
0 comments
[ 3.5 ms ] story [ 30.8 ms ] threadNo comments yet.