1 comment

[ 95.4 ms ] story [ 99.6 ms ] thread
Original title: Google AI Introduces STATIC: A Sparse Matrix Framework Delivering 948x Faster Constrained Decoding for LLM Based Generative Retrieval

Shortened to fit hn