Landmark Attention: Random-Access Infinite Context Length for Transformers (arxiv.org) 22 points by johntb86 3y ago ↗ HN
[–] version_five 3y ago ↗ They say it's infinite but then they use it to increase Llama's context length to 32k tokens instead of infinity. Still seems interesting. [–] two_in_one 3y ago ↗ Actually cool if it works. This would make many things possible on a single desktop. I'm afraid landmarking is actually a lossy compression. And the real content's size still limits model's even theoretical abilities. [–] MacsHeadroom 3y ago ↗ It's infinite but not free. Larger context still means more VRAM used and longer compute times.
[–] two_in_one 3y ago ↗ Actually cool if it works. This would make many things possible on a single desktop. I'm afraid landmarking is actually a lossy compression. And the real content's size still limits model's even theoretical abilities.
[–] MacsHeadroom 3y ago ↗ It's infinite but not free. Larger context still means more VRAM used and longer compute times.
[–] viraptor 3y ago ↗ The link to the repo (https://github.com/epfml/landmark-attention) leads to "we'll publish something later".
4 comments
[ 5.0 ms ] story [ 15.4 ms ] thread