HN Explorer

Will it be possible to do LLM Pool Training?

2 points by slicktux ↗ HN
Inspired by Crypto Pool Mining: would it be possible to train LLMs as a pool of volunteers? I feel like that would be a good way to have frontier models that are open and don’t rely on big corporations with data centers to train and release them…

2 comments

[ 2.5 ms ] story [ 17.6 ms ] thread
verdverm
possible, yes, people already have projects

practical, not yet, if ever (?)

the difference is data and calculation dependencies, in crypto they are independent

bjourne
Datacenter-scale GPGPU compute is extremely efficient compared to what you would achieve with a network of independently operated distributed heterogeneous devices spread across the world. Just imagine the overhead for ensuring that a node doesn't poison the gradients. And that's just one very small part of the problem.