Show HN: TurboQuant for mlx-lm (Apple Silicon) (github.com)

1 points by pythongiant ↗ HN
Hi HN,

I built mlx-turboquant, an implementation of Google's TurboQuant KV-cache compression algorithm for Apple's MLX framework.

The repository includes quality benchmarks, memory benchmarks, and a modular implementation so individual pieces (PolarQuant, QJL, packing, codebooks) can be studied independently.

1 comment

[ 2.6 ms ] story [ 5.9 ms ] thread
Hey guys, i'm especially interested in feedback on the kernel design and integration onto mlx lm.

Its on pypi as well as a simple pip install mlx-turboquant :P