Anyone have any predictions about when the prices will start going down significantly? Anyone think we could see price per hour per GPU cut in half by July?
Looks like we have a bunch of start-up cloud companies who have money being thrown at them by god knows who buying thousands of H100s and other GPUs. These are all going to come online around the same time in early spring. I think we will see prices drop rapidly by June .
They're not only insane (i.e. offering H100s for rent at a quarter the price they'd be to purchase) they're also impossible, since every listing claims to have deployed an Infiniband fabric speed which is firmly in the future.
Jonathan from TensorDock (https://tensordock.com/) here - we listed two of our A100 and H100 clusters on the site.
The IB equipped on our clusters (can't speak to others) is 8x 400 Gbps. Most customers training foundational models are able to fully utilize that fabric in parallel.
Which HCAs are enabling that? You're using eight 4-link QSFPs here?, presuming this is NDR?
And out of curiosity, is aggregate bandwidth the normal marketing metric in this industry? In my neck of the woods this would be reported as an NDR400 system.
Can you add the cost of the minimum reservation inline too? E.g. if it were $1/gpu/hr and "Minimum 1 GPU for 1 week" then make that text "Minimum 1 GPU for 1 week ($168)". This would allow faster comparison between options when on a budget.
That's more to rent them for 2 years than it would be to buy the gpus outright, it seems. I guess adding on the cost of all the supporting hardware might exceed it though.
Hmm who's this for? vast.ai seems far more reputable and competitive: https://cloud.vast.ai/create/ Why would I bother getting in touch with a random seller when there's a whole eco-system already in existence?
I'm trying to figure this out too. The great thing about CraigsList is that sellers are physically local, and you can haggle with them. From a seller's perspective, a physically localized audience means less competition. None of that applies here.
It's not? Craigslist is for buying other people's old used stuff. I assume that means this site is for buying old used GPUS, maybe sourced from reputable sources.
We have our own supply base (sourced through https://tensordock.com/host), operate some of our own servers, and are not related to any other marketplace :)
We think we have better security & reliability than Vast.ai due to virtualization rather than Dockerization, as well as more strict access controls [1]. Additionally, you can run Windows VMs on us if you want :)
I can almost visualize future commodity/options/derivatives trading market around underlying GPU per hour rental prices...
You know, a contract to rent a cetain GPU at a certain price per hour for a certain number of hours -- would be sort of like a commodity delivery contract.
Once those are created, then options, but more broadly, financial derivatives -- could be created on top of those...
Once those are created -- they could then be traded, held long, sold short, and otherwise "invested in" and/or "speculated on" -- depending on what terminology one prefers...
Also, to be clear, I'm not saying that this should be done -- only that it could be done...
That is (and pardon this exceptionally bad pun!) -- with respect to trading possibilities around the GPU rental prices market, you could say I am only speculating! :-) <g> :-)
41 comments
[ 4.4 ms ] story [ 87.8 ms ] threadThe IB equipped on our clusters (can't speak to others) is 8x 400 Gbps. Most customers training foundational models are able to fully utilize that fabric in parallel.
And out of curiosity, is aggregate bandwidth the normal marketing metric in this industry? In my neck of the woods this would be reported as an NDR400 system.
Aesthetics? I guess?
My first thought was, isn't Craigslist Craigslist for GPUs?
We have our own supply base (sourced through https://tensordock.com/host), operate some of our own servers, and are not related to any other marketplace :)
We think we have better security & reliability than Vast.ai due to virtualization rather than Dockerization, as well as more strict access controls [1]. Additionally, you can run Windows VMs on us if you want :)
[1]https://tensordock.com/security
https://twitter.com/natfriedman/status/1757923480434323521
/s
You know, a contract to rent a cetain GPU at a certain price per hour for a certain number of hours -- would be sort of like a commodity delivery contract.
Once those are created, then options, but more broadly, financial derivatives -- could be created on top of those...
Once those are created -- they could then be traded, held long, sold short, and otherwise "invested in" and/or "speculated on" -- depending on what terminology one prefers...
Also, to be clear, I'm not saying that this should be done -- only that it could be done...
That is (and pardon this exceptionally bad pun!) -- with respect to trading possibilities around the GPU rental prices market, you could say I am only speculating! :-) <g> :-)
Anyway, thumbs up for GPUlist!