[–] dane-pgp 5y ago ↗ > What are those 175 billion parameters?> According to Geoffrey Hinton — the godfather of ‘deep learning’ — parameter can be thought of as a synapse in a human brain ...> They’re far, far smaller, the biggest ones we have right now have about a billion synapses. That’s about a million times smaller than the brain.So his estimate is now off by a factor of about 100? How long will it take for the next couple of 100-fold increases in parameter count?
1 comment
[ 3.3 ms ] story [ 10.8 ms ] thread> According to Geoffrey Hinton — the godfather of ‘deep learning’ — parameter can be thought of as a synapse in a human brain ...
> They’re far, far smaller, the biggest ones we have right now have about a billion synapses. That’s about a million times smaller than the brain.
So his estimate is now off by a factor of about 100? How long will it take for the next couple of 100-fold increases in parameter count?