HN Explorer

8 comments

[ 3.2 ms ] story [ 35.9 ms ] thread
nalzok
> at market price $600k

I don’t think they are really spending $600k. Chances are that they own a GPU cluster so they don’t need to pay the cloud premium.

fragmede
Spot instance pricing for p4d.large is about a third the on-demand cost, though it's not clear what they're using to calculate market rate. An A100 card is roughly $10k retail, so it's not like "owning a GPU cloud" is that cheap a proposition either.
Slix
Could training be crowd sourced among consumer GPUs like Folding at Home?
angry-tempest
Probably not for a few years, you need a (maybe few) A100(s) to be able to backprop a model that big with float32.
verdverm
iirc, they tweeted about using around 3800 in parallel
jacooper
Is that relatively cheap or expensive?

I have no context for this, do other model dveelopers publish how much it costed to train them?

hnfong
Given the impact the model might have on the world, and that $600k is roughly equal to the annual salary of 1~3 top notch engineers... it's probably intended to mean relatively cheap.

For similar context, millions of VC money have been thrown around for much less impressive results.

#firstworldproblems