Isn't Blackwell meant to be training oriented? And being able to run inference on the same architecture is a nice bonus?
Groq is only inference. Of course, if you build an ASIC specifically to run inference, it's going to be faster and more efficient. But things are moving so fast that by the time Groq gets any kind of market share, perhaps newer models will not be compatible with the ASIC?
The cost advantage of Groq doesn't seem like it's worth switching over. It needs to be 10x better in cost to get companies to make the huge effort to switch?
4 comments
[ 2.1 ms ] story [ 20.9 ms ] threadGroq is only inference. Of course, if you build an ASIC specifically to run inference, it's going to be faster and more efficient. But things are moving so fast that by the time Groq gets any kind of market share, perhaps newer models will not be compatible with the ASIC?
Can anyone speak to this?
Also, swyx did a cost analysis between Groq vs H100 here: https://twitter.com/swyx/status/1760065636410274162
The cost advantage of Groq doesn't seem like it's worth switching over. It needs to be 10x better in cost to get companies to make the huge effort to switch?