Yeah, and its a waste. Nvidia runs the A100s in a relatively inefficient power band (300W or 400W), and tons of power is burned on the interconnect huge LLMs need to fit in memory.
And the servers cost a fortune, with a huge profit margin.
Its not sustainable... in fact, its probably less efficient than the crypto mining boom, as miners were downclocking GPUs (and building simple ASICs) to run at more efficient voltages.
4 comments
[ 3.6 ms ] story [ 19.4 ms ] threadAnd the servers cost a fortune, with a huge profit margin.
Its not sustainable... in fact, its probably less efficient than the crypto mining boom, as miners were downclocking GPUs (and building simple ASICs) to run at more efficient voltages.
It seems like Nvidia made a bajillion V100s (and some P100s) but very few ended up on eBay and such. I would have liked a cheap one.