Running LLMs with 3.3M Context Tokens on a Single GPU (arxiv.org)

Their demo looks really cool: https://github.com/mit-han-lab/duo-attention