> that aims to help machine learning engineers <word is missing in text> more efficiently
Seems like a PR template went wrong.
OK, further down they explain, that it can start clusters and automatically parallelize deep learning model training across them. I thought that would require actual research, as many models require different hyperparameters when spread out over many nodes.
well... that's part of the magic we're bringing to this. PyTorch Lightning is built exactly for this purpose, so it is highly scalable and designed for working at this kind of scale.
I built it while doing self-supervised learning research at Facebook AI research where I was running dozens of model versions each on 256 GPUs!
We've applied a lot of these lessons to Grid, so when you use Lightning with it you have to do nothing. For other frameworks you also don't have to do much in most cases but if there's some friction there we have tools and guides to help!
4 comments
[ 3.1 ms ] story [ 20.9 ms ] thread> that aims to help machine learning engineers <word is missing in text> more efficiently
Seems like a PR template went wrong.
OK, further down they explain, that it can start clusters and automatically parallelize deep learning model training across them. I thought that would require actual research, as many models require different hyperparameters when spread out over many nodes.
I built it while doing self-supervised learning research at Facebook AI research where I was running dozens of model versions each on 256 GPUs!
We've applied a lot of these lessons to Grid, so when you use Lightning with it you have to do nothing. For other frameworks you also don't have to do much in most cases but if there's some friction there we have tools and guides to help!