Sparse Llama: 70% Smaller, 3x Faster, Full Accuracy (cerebras.net)

40 points by panabee 2y ago ↗ HN

1 comment

[ 1.9 ms ] story [ 12.9 ms ] thread

Specifically this is Llama2, not Llama3, was a bit disappointed from that. Also wasn't totally clear from the article - will this actually increase GPU inference speed / decrease GPU memory usage?