Skipping 90% of KV dequant work speeds up LLM decode by 22% (github.com) 1 points by pidtom 3mo ago ↗ HN
1 comment
[ 2.7 ms ] story [ 12.2 ms ] thread