From multi-head to latent attention: The evolution of attention mechanisms (vinithavn.medium.com)

Do we know if any of these techniques are actually used in the so-called "frontier" models?

Any way to read this without making an account?

"Attention Is All You Need" - I've always wondered if the authors of that paper used such a casual and catchy title because they knew it would be groundbreaking and massively cited in the future....

[dead]