HN Explorer

Ask HN: ML models trained on decentralized computing?

1 points by frozencell ↗ HN
Could models such as Dalle2 (huge amount of parameters), be trained on distributed/decentralized network of volunteer/paid computers in the style of LHC at home [0]

[0] https://en.wikipedia.org/wiki/LHC@home

1 comment

[ 3.5 ms ] story [ 13.9 ms ] thread
verdverm
what does backprop look like? how long does a single training iteration take? what happens if someone drops out mid loop? how much does it cost to move the training data around?