KVarN: Native vLLM backend for KV-cache quantization by Huawei (github.com)

it should be easy to do btw

Last I heard, vLLM was backed by a company that has raised $150m in seed funding. I'm sure they've got the resources to port it.

Why this is not a PR for llama.cpp

[dead]

Faster than Fp16, not better quality i guess

It's not better quality: 59.3% vs 59.4% fp16 on AIME 25

any divergence (even if the benchmark is better) from full precision is error