> When a LLM outputs a word, it commits to that word, without knowing what the next word is going to be. Please, people, read before you write. Both the article and the paper explain that that’s not how it works. ‘One…
Only starts approaching PPL of AR at block size 4. May as well just use multi-token prediction with a standard AR model.
> When a LLM outputs a word, it commits to that word, without knowing what the next word is going to be. Please, people, read before you write. Both the article and the paper explain that that’s not how it works. ‘One…
Only starts approaching PPL of AR at block size 4. May as well just use multi-token prediction with a standard AR model.