HN Explorer

Show HN: gpudeploy.com – "Airbnb" for GPUs (gpudeploy.com)

267 points by nicowaltz ↗ HN
Hi HN,

YC w24 company here. We just pivoted from drone delivery to build gpudeploy.com, a website that routes on-demand traffic for GPU instances to idle compute resources.

The experience is similar to lambda labs, which we’ve really enjoyed for training our robotics models, but their GPUs are never available for on-demand. We are also trying to make it more no-nonsense (no hidden fees, no H100 behind “contact sales”, etc.).

The tech to make this work is actually kind of nifty, we may do an in-depth HN post on that soon.

Right now, we have H100s, a few RTX 4090s and a GTX 1080 Ti online. Feel free to try it out!

Also, if you’ve got compute sitting around (a GPU cluster, a crypto mining operation or just a GPU) or if you’re an AI company with idle compute (hopefully not in a Stability AI way) and want to see some ROI, it’s very simple and flexible to hook it up to our site and you’ll maybe get a few researchers using your compute.

Nice rest of the week!

194 comments

[ 2.5 ms ] story [ 242 ms ] thread
malfist
I'm curious how a business pivots from drone delivery to GPU Airbnb?
exe34
I got it from the line about how they trained their robot models - think of Amazon pivoting to aws, the substrate becomes the commodity. Unless you're asking if it makes sense, or what went wrong? Then I don't know. I imagine startups that survive usually pivot from something else?
MikeTheGreat
I've never run a start-up (successful or otherwise :) ) but I've also heard that being able to pivot can be really useful - Flickr is often cited as an example.

Also, "GPU on demand" sounds _a_lot_ easier than "drone-based delivery". Between Seti@Home/Folding@Home/etc, various grid-computing/clustering/orchestration stacks that already exist, etc it seems reasonably doable to implement in a year or so. "drone-based delivery" sounds capital-intensive, sounds like you'll need to spend a lot of time building a professional network of business people who might use the service (so there's a 'cultural friction' between techie founders and business folks, potentially), plus the ever-looming threat of Amazon/etc figuring this out first.

tl;dr: I agree it's weird pivot, and good on the founders for being able to make the change! :)

jayyhu
To add to "capital-intensive": the regulatory framework right now (in the US) for drone delivery is uncertain at best. While most of the rules & regs to enable drone delivery is already finalized & published, a few very important ones (such as BVLOS - beyond visual line-of-sight) have yet been finalized, and thus need one-off waivers from the FAA to allow. And getting those one-off waivers is what eats a lot of time & money.
nicowaltz
While most people think it's cool and some think it's scary, drone delivery is ultimately something that people (that is: companies with money) don't need.

Turns out, using drones to deliver is also not that competitive either. Delivery vans are very cost-efficient, and for food delivery / on-demand delivery, drones are not able to carry most orders. So it's not even the regulatory pains that make this difficult, which are unbearable in their own right.

It was a lot of fun to work on and we would have definitely stuck with it if there was any interest. There was none, so we had to admit that to ourselves.

This is a hard pivot, but it's been very stimulating to work on.

qeternity
Btw thanks for this honest insight. When I read the bit about your pivot, I sort of rolled my eyes. But this is a really nice, sober reflection that actually builds credibility with people like me who might make snap judgements at first.

I hope you guys find traction.

callalex
Their parents are good friends of someone else’s parents.
a2128
What's so unique about this compared to just using RunPod?
yeldarb
Love the concept.

I've used vast.ai (similar "Airbnb for GPUs" pitch) for years to spin up cheap test machines with GPUs you can't really find in the cloud (and especially consumer-grade GPUs like 4090s). Any insight into how this is different/better?

ganoushoreilly
Im also interested in what the differing factor is. Would also like to see more documentation for onboarding rather than just "Ubuntu and root available".
icelancer
vast you have to choose specific machines. gpudeploy routes to whatever resources are available.

vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way. Someone on HN wrote a script to run a test docker image on every machine and auto-tagged the machines' quality using their API, which is what I'd do if I was going to use vast seriously for compute.

lelanthran
>> Im also interested in what the differing factor is.

> vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way.

Wouldn't gpudeploy have exactly the same problem? How is it mitigated with gpudeploy?

everforward
I think it's more of a business strategy issue than a technical one.

I suspect it would be trivial for Vast or GPUDeploy to spin up a benchmarking job before allowing sales on that machine. I'm not an expert on PCIe lanes, but I would think the performance issues would be visible via bandwidth or latency on the lanes.

It kind of makes sense to me, though. If I were looking for absolute reliability and was willing to pay for it, I'd just go to one of the many GPU cloud vendors. Likewise, I suspect anyone willing to really work on getting good performance would rather be a real provider or sub-provider than being part of this nebulous C2C GPU cloud.

polygot
Do you have a link to that thread/Docker image? I would be very interested using it
icelancer
I don't, sorry. I would love to use it as well, I should've bookmarked it! Also, I'm not sure the person opensourced it.
firloop
I use vast.ai somewhat often. It's great!
tehsauce
+1 for vast. they usually are the cheapest and have the most supply. some instances can be less reliable at the low end though
nicowaltz
Main difference is that we are more opinionated (in terms of configurations) and sort of do the scrolling and sorting out for you – hopefully a bit smoother as a user experience. We sort out bad machines immediately. We're also directly working on making compute from unknown high-end data centers available, there's a lot of unused compute out there! See gpulist.ai

Also, don't know if vast.ai does this, but with us you can have 6 user sessions on your machine if you have six GPUs, so granular utilization is possible.

bradfox2
We own and operate 40+ data center GPUs (v100s, a100s, and ax000s) in a private cluster and use vast to rent unused capacity.

What would make you better than vast is extremely easy spot leasing and job prioritization.

I want to be able to have one of our training jobs finish, and then have the capacity immediately transition to a lease. With vast, we are renting in week long blocks.

bigcat12345678
You can do that on llm.sxwl.ai Shoot me an email at z@sxwl.ai for instructions, the web site is pretty outdated, the main UI is through restful API (which we don't have time to write doc yet)
mkl
If you have time to answer emails with instructions, you have time to update your site and documentation. Why not just do that?
grepfru_it
Don’t build something if you don’t have a use case. All you have is a wishlist, until someone says “yes I want this here is $$$” which I assume the email will facilitate.
DeathArrow
Can you expand on spot leasing and job prioritization? What kind of api would you prefer? How would you like to adjust time slices?
jcannell
You should be able to do that right now on vast. You just need to rent the gpus yourself with your own on demand instance(s) for your training job. As soon as it finished you then stop or destroy those instance(s) and the GPUs are available immediately (and if there are any other instances queued up in scheduling they will start up). Your actual job doesn't necessarily need to run in the container (if you know what you are doing).

(I'm the founder of vast btw - contact us for help on setting this up and/or any feedback on making it an easier/better process)

cfn
Just a quick comment: The country list in your Add Basic Node Data is not sorted.
idiotsecant
This is an interesting idea but it would be cool if it were more granular, like I pay while my payload is executing only, and an API abstracts away which GPUs I'm running on, the execution environment, etc and just let's me push in code and get out data when it's done. Maybe that's what this is, not sure. I hit an account sign up before I could figure it out.
dbish
I think products like this have to evolve into pay by the hour otherwise you find alternatives that have it.
spxneo
doesn't seem all that cheap compared to whats available
bllchmbrs
For those that wants prices checkout: https://gpumonger.com/

This seems much more in depth, and a true service, but for those that just want to compare prices check out gpumonger.

latchkey
There is no 'contact us' page, which is a little concerning.
Ocha
They are not providing actual service - just price comparisons across different providers. Why is contact us page missing a concern here?
latchkey
I'm a provider, I'd like to list my service. How do I do that? Since there is nothing obvious, then how do you know the providers they list are a valid representation of what is out there? Maybe there are other providers who don't want to pay to be listed? But how do you even pay?
mike_d
They list over 40 vendors. If you want to be listed, maybe get bigger?
LoganDark
Who says they're even looking? Being able to contact them would tell you right away whether they simply overlooked you or whether you don't yet meet some criteria (which they might actually tell you).
trogdor
I have read many of your comments on GPU-related posts. Occasionally, when I see your comments, I visit your website to see if I can learn anything about your business beyond what you are posting on HN. To this day, hotaisle.xyz contains nothing but a logo.

I don’t get it.

How can you expect a website to list your service when your own service’s website contains zero information? Why would you pay to list your service, when there is no information available about the service you provide? Am I looking in the wrong place? So confused…

latchkey
> hotaisle.xyz contains nothing but a logo.

To be fair, there is also a contact email at the bottom of the page.

> How can you expect a website to list your service when your own service’s website contains zero information?

We are the first and only (for now) verified MI300x provider on gpulist. In order to get verified, I contacted them directly, they asked for a few bits of information about my business, including my EIN. What I'm offering there, is exactly what I have today.

https://gpulist.ai/detail/3c18f8a

> So confused…

I know, it is ok. Let me explain a bit. We are starting small, so the website is the last focus right now. I know the general expected culture is to have some splashy page with a typeform on it, but hey... aren't we also a bit tired of that?

In order to even get access to buy these GPUs, you have to go through quite a lot of effort. You can't just buy them off the shelf from BestBuy. They are export controlled and I've agreed to not use them to build bombs. You have to have a valid business and a great story, or they won't even talk to you. Heck, I even had to prove my business was in good standing in Delaware. I'm pointing this out because I will need to know all my customers too. My business isn't something someone just signs up for on a website.

These GPUs are also extremely expensive. Imagine a 350lbs Ferrari. We started with 8 of them (one chassis) because they are super new and it was a proof of concept. Last year, we didn't even know for certain if AMD would double down on AI. This is all we initially raised funding for. As soon as we deployed the compute, we immediately had a customer on them, all without a website. Just word of mouth. By the way, the success of the PoC unlocked our next round of funding, and we are working on a much larger order of MI300x right now.

Don't worry, you'll get a website at some point. That said, these things sell themselves, you either have them or you don't. I've been very transparent and public about what we are up to. Would a website really help here? Maybe. But I've also started other extremely successful businesses originally without websites too. At the end of the day, I'd rather spend investors money on buying more compute, than a pretty website. Once I have some more revenue, I'll funnel that right back into the business and work on marketing/sales more.

If you're curious about anything, feel free to just reach out and ask. I'm not some corporate overlord suit wearing sales guy. I'm an open source tech nerd who's been in the business a long time. 20+ year ASF member, who co-founded Java @ Apache. Happy to answer any questions.

trogdor
Good luck with your business!
latchkey
Thank you, we need all we can get! Super excited about it though. Big things coming in the pipeline.
bllchmbrs
This was my project that I recently sold. If you want to be listed I can put you in contact with the folks that bought it. LMK!

The lack of "contact us" was pure laziness at the time.

latchkey
Congrats on the sale! Interesting to make a few pages and then sell it.

Sure, assuming the site will continue... my email is in my profile.

krasin
Related: https://vast.ai is good and cheap. Just don't put any sensitive data on these GPU machines.

I've been a happy user of vast.ai for some time now.

overgard
Pretty neat, but is there any Windows or MacOS support planned? I wouldn't mind renting out my GPU when it's idle, but I don't really want to go through the process of dual booting etc.
bongodongobob
What are you doing GPUs that requires Windows or Mac support?
zephyrthenoble
Video games
otterley
Have you looked at Nvidia GeForce NOW? It's like $10/mo for a pretty decent streaming gaming rig. I'm very happy with it - I don't have to deal with Windows and can play AAA games on my Macbook Pro at 60Hz (1080p).
erhaetherth
> on my Macbook Pro at 60Hz (1080p).

I think you just answered yourself. Some of us like to play games at 4K at 80Hz+, with no subscription fees, no internet bandwidth requirements, no added latency, and ability to mod.

hhh
Yeah, but that has nothing to do with the context of the question. Someone is specifically asking about accessing a GPU over the internet for video games.
freedomben
Nowadays it's rare that a Windows-only game that I want to run doesn't run flawlessly on my Linux machine (through Proton/Wine). I wouldn't recommend going outside of Steam though unless you're willing to do some troubleshooting.
1992spacemovie
Try running COD Warzone on Linux and not get quarantined to cheater-suspect lobbies.
freedomben
Ah yes, apologies that's an important caveat I forgot to include.

Games that use anti-cheat are a mess on Linux. I don't play any of those games, and if you do then you're likely to run into some trouble with Linux-only.

1992spacemovie
Yeah anti cheat sucks overall tbh. I loathe giving kernel level access to a random video game’s anti cheat system.

Short of dedicated hardware (Xbox/ps) I’m not sure what else could be done.

bongodongobob
So what are you looking for, an RDP session with an attached GPU?
nicowaltz
Windows support is definitely something we are thinking about.
EE84M3i
What's the security story here?
htrp
probably very basic... so don't run it on anything that has your own data on it (if you're an AI startup, definitely don't run it on your research cluster).
Traubenfuchs
> definitely don't run it on your research cluster

...what‘s the threat, actually? GPU time sellers stealing your secret sauce?

lolinder
I think they mean don't lease out your research team's GPUs and allow random people to run untrusted code on your cluster, lest they figure out a way to break out of any sandboxing the software has in place and get loose in your network. The company's current answer to that concern is "everything runs inside Docker, configured with security best-practices", which is less than inspiring.

https://news.ycombinator.com/item?id=40261591

remram
What's preventing GPU providers from sending wrong results instead of actually running the computation? For example, send the last computed result? Is this something that the renter has to handle by adding their own checks?

In addition to the problem of the renter crashing your machine or reading your password through DMA, of course.

latchkey
What incentive would a GPU provider have to spend time figuring out what result to send for some custom application?
remram
The incentive is huge, if I spend 2 milliseconds sending you your previous results instead of 2 hours running your new computation, I can (pretend to) run way more computations on the same hardware and collect hundreds of time more money.
latchkey
At the risk of being exiled off the platform and earning nothing. Don't forget, there is a bit of KYC with Stripe.
krapht
ID verification before you can host and random audits from gpudeploy.
remram
That's my whole question, do they do random audits, or is it the job of customers to double-check their results for possible attack or compute-theft and report.
greenish_shores
It seems wrong to call it a "job of customers". It's like you wrote a Bitcoin client which didn't verified hashes of transactions, "trusting" everything. Or like serving a website with login feature supporting only HTTP, not HTTPS. It is a very basic feature of whatever software would connect to such services.
remram
So it is the job of the customer to write their own Bitcoin or HTTPS client, in your metaphor.
greenish_shores
Every technology was (very) underdeveloped at some early point in its evolution.
remram
That's what I asked, how developed is it now. Why so defensive?
greenish_shores
I don't know how developed is it now, I'm not associated with the startup shown in any way. It's mainly a question to them. However, in terms of wider industry, in general distributed high-performance GPU(-like) computing "for everyone" is in its infancy. 99% of what was already done up to this point was targeted to people who would both buy and supply power "in bulk", not "in retail". Perhaps with a little exception of several excellent projects like Folding@Home and other @home's.
greenish_shores
NO. That's the worst way to do almost anything on the Internet, and should be considered a last-line defense, if nothing else can be done. Here, it can be. See my comment above.
greenish_shores
Run 1/10,000 - 1/100,000 of computations locally, and also send them as tasks to be send remotely. If compare yields difference, repeat both. After, say, 10 tries, blacklist the provider. Of course it will take a lot more nuances to implement that, but that's the general idea. It's a no-brainer.
remram
Sounds like a lot of work that I would expect the paid service to help with.
[deleted]
(comment deleted)
greenish_shores
Linux supports IOMMU on most platforms.
remram
I fail to see how this relates. If you can't trust the provider, why does it matter whether they say they have IOMMU or not.
lschneider
At the moment, we manually verify operators and are currently onboarding some tier-4 operators. Down the line, we'll have a 2-tier system where you can choose whether you want a verified machine or not. From the operator's perspective, everything runs inside Docker, configured with security best-practices.
lolinder
I've always understood that containers are not proper sandboxes and shouldn't be used for containing untrusted code, no matter the best practices used. Has this changed in recent years? Do you have documentation for what sorts of best practices you're using and why they are sufficient for executing untrusted code?
gavindean90
You are correct from my knowledge. I would expect that if the container is set to not run as root you might be able to enforce fine meaningful security but I’d still run it in a VM if feasible.
janosdebugs
Having done a little bit of work in the area[1], I think you should publicly document exactly what those best-practices are. Are the workloads running in a networkless container? Do you limit IO? Do you limit disk usage? Answering these in detail would help you gain customer trust on both sides.

[1]: https://containerssh.io/v0.5/reference/docker/#securing-dock...

flaminHotSpeedo
So you don't have real security for operators, is what you're saying.

Containers are not, and will never be, a secure isolation boundary.

callalex
Back in my day we cooperated to advance medicine and science with folding@home. How times change.

https://en.m.wikipedia.org/wiki/Folding@home

EduardoBautista
folding@home was used in the “console wars” to justify one’s choice of the PS3 over the Xbox 360.

Was the cell processor in the PS3 really that efficient for this purpose?

TkTech
It was a fairly unique architecture that had some pros and cons. In reality what made it a winner was the steep subsidy by Sony (units were sold at a loss, making the assumption you would buy a few games) and the availability (later removed) of an official Linux distribution.
luigi23
very few cared about linux. it was mostly hd-dvd (later dvd) vs blu-ray.
kuschku
It was basically an all-purpose vector computing monster focused on SIMD. You could use it for physics simulations, animations, tesselation, etc. Basically everything you'd use a compute shader for nowadays.

That's why emulators need AVX512 support to match the PS3. It was incredibly powerful.

Obviously, in that era's single-threaded world no engine could make use of that functionality and few knew how to program for it. It was ahead of its time, by quite a while.

fl0ki
Remember around that time when the industry said that OpenCL would allow write-once run-everywhere compute code for a booming industry of diverse and competitive compute devices? I fell for that scam for a couple of years, before very fortunately getting a different job for long enough to watch the collapse from a safe distance.
AbstractH24
Thought the same thing when I read this.

How times have and haven't changed.

mattxxx
Doesn't this just raise the price of gpu's, and further encourage people to hoard them?
nirvanatikku
Great idea and wishing you the best of luck!

Dropping a note that I've found https://akash.network/ ~ https://akash.network/gpus/ to be impressive, as typically with crypto projects it's all scams, however in this instance there's demand and legit usage. https://stats.akash.network/

Something to consider!

lordofgibbons
I was excited to see their offering, but unfortunately they don't take fiat - only some specific cryptos.
barbarr
I feel they could onboard so many more users by having a fiat frontend that doesn't refer to blockchain. But maybe they're targeting crypto natives so users don't get shocked by price variability etc.
david_shi
you can use akash gpu's on brev.dev
densh
Are you planning on letting hobbyists rent their single machine with multiple GPUs?
nicowaltz
Definitely, but we will sort out machines that are unreliable quite quickly.
thih9
> or if you’re an AI company with idle compute (hopefully not in a Stability AI way)

What does that mean? I’m likely missing some context, could anyone explain?

latchkey
Interesting, these companies are springing up left and right. Software solutions to a hardware problem. It looks like most of the available compute is allocated, so they are going to have to get a bunch more providers onboarded.

I wonder why y-combinator is stuffing their investments with multiples of these similar companies... https://www.shadeform.ai/ is another one.

A few quick comments:

Reading the source of their install script:

https://gpudeploy-public.s3.us-west-1.amazonaws.com/join_clu...

It doesn't start off with set -e, which could result in an incomplete install, yet appear to finish.

It also installs some binary "instance-server"... who knows what it does... would you trust this on your server on your network?

It is nvidia specific... sadly, don't expect AMD gpus anytime soon.

Feels like a MVP, let's see how this grows over time.

dheera
Is there any way to monitor for sales of hardware if/when any of these companies dies so that I can get 1xH100 for personal use at a deep discount?
latchkey
Ebay?
dheera
Is that where they really go? I feel like buying anything for several thousand dollars on Ebay is sketch. What if they ship me an empty box?
latchkey
Back in my crypto asic mining days, I'd hear stories of people receiving boxes of cement blocks.

Looks like you can get a MI210 on ebay for $6500, with 30 day warranty. heh.

Oh, there are a bunch of H100's there...

15155
File a dispute with PayPal? And then with your credit card?
tptacek
As the post points out, this is a pivot from a company that was a consumer of this kind of service in its original incarnation; YC had presumably nothing to do with it.
latchkey
That an interesting presumption. If a startup is "Backed by Y Combinator", and it pivots, you don't think that Y Combinator would have some sort of opinion on that?

I would expect some sort of conversation like: "Oh hey, we have another company, in our portfolio, that is far ahead of you, doing exactly the same thing. Maybe you should do something else?"

But of course, this is AI... plenty of space for gpu marketplaces.

dang
YC partners might have an opinion and might offer advice but they're always going to support what the founders want to do—that's kind of the core principle of YC.

As for "other company in portfolio", that's unavoidable when funding thousands of startups and almost always turns out to be a non-issue.

latchkey
Awesome, makes perfect sense. Thanks for the clarity @dang.

I'm curious now, if you can say. Was advice offered in this case? If so, what was it?

dang
I'm not part of such discussions so I'm afraid I have no idea!
2024throwaway
People here think you _are_ YC. Little do they know you're far more perceptive and capable than YC.
dang
That's definitely not true, but thanks for the support :)
2024throwaway
Truth is a spectrum. I will leave my opinions on YC unpublished. Thanks for all you do.
tptacek
That's not how YC works. We're YC W20. We have a GPU offering. Guess how many conversations we had with YC about it.

What's YC going to do? They have a tiny stake in your firm. That's the point; that's what "founder-friendliness" means. They're not your board.

latchkey
Cool, thanks for the info Thomas. I honestly have never looked into YC at all, so I'm definitely not aware of how these things work. I was posting from an honest curiosity.
moneywoes
why does ycombinator invest in companies that will compete against each other, isn’t that a conflict of interest
gadflyinyoureye
Nope. That allows them to control the market. Find new ideas that could threaten the big 5 status quo. Fund them. Get them bought by one of the big five (making a good amount of money off that). Let the big five shutdown the competition.

Conflict of interest only applies if you think they’re concerned with the public good. There is no assumption of that here. Every man for himself.

scottfits
I have a cluster on RunPod and it's great, but there's definitely some opportunity in the space, for example if you can focus on transparency (utilization rates, more info about end user and their purpose, reporting) that would be helpful.

I think RunPod and Vast have most the market share but t's still early to the game

animex
I wonder if I could get strangers to heat my home with GPUs in the winter...
modeless
How are you different from vast.ai? What's your payout rate?
oefrha
I don’t get “final rate is usually lower”. Why is it lower, and by how much? And since it’s “usually”, what about the unusual case? Equal to the sticker price, or higher? But since you use “<” neither should happen? I would expect an exact price tag for a “no-nonsense” pricing scheme. Right now it’s just confusing.
nicowaltz
Thanks for the input, you're right, maybe a fixed price is better. The idea currently is that the quoted price is an upper limit on the rate you pay and the final rate depends on what the GPU provider inputs, which will normally result in a lower rate than the upper bound. At the moment we have upper bounds if you want to be included in on-demand input, and also depending on the service you provide (security, network etc.). We determine pricing with partners 1-on-1 a the moment.
latchkey
I would focus on offering a pricing model that entices people to want to pay.

I've found that usually this involves something extremely simple and easy to understand, with clear up front numbers.

oefrha
Thanks for the answer, probably a good idea to include some version of this in the footnote.
CtrlAltDelete51
What kind of utilization percentages do you expect to be able to provide? Do you have any existing usage/ROI data?
nicowaltz
Just launched but will put a panel on the landing page for that, once the data comes in.