1 comment

[ 4.7 ms ] story [ 16.5 ms ] thread
The Dive into Deep Learning book is the first book I've seen with a chapter dedicated to the Attention Mechanism.

Attention is central in recent breakthroughs like AlphaStar and AlphaFold, and a major omission from other books, even those published in 2020.