I made this interactive reference for transformer models, showing everything down to elementary math. I intentionally avoided matrix multiplication and used explicit sums and indices instead. Covers models from GPT-2 to Qwen 3.6, with MLA, MoE, RoPE, MTP, etc togglable. It's best viewed on desktop or tablet.
2 comments
[ 3.1 ms ] story [ 18.6 ms ] thread