Nonlinear Attention Decoded – Verified by 3 AI Systems (zenodo.org) 1 points by GhostDrift 11mo ago ↗ HN
[–] GhostDrift 11mo ago ↗ Most AI models blend meaning — but this one selects.GD-Attention is a provably nonlinear attention mechanism with: ・ No softmax ・ No averaging ・ A unique semantic jump point $s^*$Verified independently by Gemini, GitHub Copilot, and GPT-4. → Softmax isn't just suboptimal — it's structurally incapable of what this model does.
1 comment
[ 3.9 ms ] story [ 12.4 ms ] threadGD-Attention is a provably nonlinear attention mechanism with: ・ No softmax ・ No averaging ・ A unique semantic jump point $s^*$
Verified independently by Gemini, GitHub Copilot, and GPT-4. → Softmax isn't just suboptimal — it's structurally incapable of what this model does.