Launch HN: Snark AI (YC S18) – Distributed Low-Cost GPUs for Deep Learning
We are Sergiy, Davit and Jason, founders of Snark AI (https://snark.ai). We provide low-cost GPUs for Deep Learning training and deployment on semi-decentralized servers.
We started Snark AI during our PhD programs at Princeton University. As deep learning researchers we always experienced lack of GPU resources. Renting out GPUs on the cloud didn't fit in our budget, and purchasing GPU cards was difficult -- at that time, so many GPUs were being taken away by the crypto-miners. Then we found out that GPU mining profits lag far behind public cloud GPU prices.
On top of that, we figured out that there's a way to run Neural Network inference and crypto-mining simultaneously without hurting mining hash rate. This observation is a little counterintuitive, but it turns out that anti-asic hashing algorithms are designed to be extremely memory intensive, which leaves a good chunk of the CUDA cores idle. We can utilize the leftover compute power to run Neural Network inference extremely cost efficiently, which could be a life savior for large-scale inference tasks. http://snark.ai/blog
At the same time, we provide low cost raw hardware access for Neural Network training. We aim to be up to 10 times cheaper than on-demand instances on public cloud, undercutting preempteble/spot instance by up to 2x. When the GPU is idle our algorithms efficiently switch to mining to reduce costs. Try it out at https://lab.snark.ai, with 10 hours of free GPU time. We made it very simple to access the hardware through a single command line after `pip3 install snark`. More information on usage here https://github.com/snarkai/snark-doc. We are also working on creating a hub for NNs, similar to docker hub. It is still work in progress but you can take a look at couple examples at https://hub.snark.ai/explore.
We would love to get your feedback, to understand how was the experience for training Deep Networks through our platform and then deploying.
69 comments
[ 4.7 ms ] story [ 134 ms ] thread(All values in $ I assume)
0.25x24x30.5=183
For 100 Eur, or about $120, you can get a 1080 inside A DEDICATED SERVER (!!) at Hetzner: https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...
I guess I have a business idea then: charge 0.5/h to rent the 1080, pocket the $63 difference per month and call it profit, undercutting Snark by 33% without even doing any crypto mining or anything on the side.
Or, just call it step 1 for mega profits! Step 2: resell the CPU computing power, step 3: resell the SSD storage space, step 4: resell the bandwidth, etc. (not sure you can resell the unused RAM, but that's another "innovative business" waiting to happen!!)
Compared to renting servers and properly configuring them (anycast, geoip, etc) I often fail to see the value that "distributed" or "cloud" offer provide besides fast scalability.
It is nice to be able to put 4x more GPUs online in a few hours instead of a day, but I am not sure it commands a 33% premium except in very specific marginal cases.
2 years is a Moore's Law doubling, which I've found tends to mean hardware offerings will be different. They're not necessarily dramatically different, if there's no new/unmet market demand, but this was a noteworthy enough one that I was (and still am) touting it as an advantage of own-hardware over cloud infrastructure.
I do not mean that in a bad way, just in a logical way.
I mean, they are already making a profit at $120/month, so I guess their response to you will be "sure, buy as many servers as you want, price is $120/each". You will be back to square one, trying to sell your hourly services to scientists who need GPUs.
Then the initial problem remain: anyone with a baseline demand for GPUs is better off renting them at hetzner. They can use you for small loads, or unanticipated needs. But then it will be for a short time, before they opt for a monthly rent.
Even then, for this peak demand, you compete straight on against google cloud and aws. You certainly undercut their ridiculous prices, but it is not clear to me how better off I am chosing you compared to hetzner+any other cloud offering.
I am just talking as a prospective client (I often need GPUs!) who fails to see what's unique or interesting in your offer. And if you know less about your competition than your prospective clients, I see that as a bad sign: your offer may not be priced right.
Maybe I am wrong, and you are just aiming for a different kind of clients, with a time-sensitive but less elastic demand, yet not as inelastic as someone who will pay top $ for google or aws? Feel free to explain me if there is business secret at risk here.
Good luck anyway!
We are experimenting with pricing and if you want to rent for a whole month our price will be cheaper than that, email us. Just thinking about efficient utilization, you might end up paying less if you don't have 24/7h jobs running.
At the same time, we are building software stack to utilize these hardware efficiently for Deep Learning applications. We need those resources for offering higher level ML products.
Anyway, the more competition the better, and I'm sure you will find a place!
I suspect that, in general, it commands an even higher price premium, as irrational as that may seem (or actually be).
This particular business is based off the notion of fixed budgets (and, presumably, short time limits), which means that, no matter how much cheaper over all it is to rent resources for a full month, it's worth the premium to rent as much as you can for a result in 10 days.
This reasonining can apply to an early startup, too (in terms of time-to-market and unpredictable scalability), but it becomes actually-irrational when it's not re-evaluated and a huge premium [1] is being paid for routine, easily-predictable infrastructure sourced from cloud providers.
[1] hundreds of percent, i.e. multiples
That being said, cloud hosting is also funny in a different way: it reminds me of the late 90s, when you had to declare your hostname and use a ftp account to upload your files to your host.
https://www.perkinscoie.com/en/news-insights/california-upda...
I don't think that extends to full account deletion, though, more about stopping recurring charges.
http://www.datacenterdynamics.com/content-tracks/servers-sto...
I wonder how they plan to undercut nicehash?
From https://cloud.google.com/compute/pricing it looks like the non-pre-emptible K80 pricing is $0.45 USD per GPU.
Can someone from Snark correct me if I'm wrong?
For some reason, the snark.ai homepage brings my laptop to its knees though. Do I need a GPU cluster to see it?
Anyhow: Can I use snark.ai to run WebGL in a browser?
Regarding WebGL, actually that is an interesting point, would like to know more about the use case.
Is storage persistent? Will files get deleted when I stop a machine?
http://forums.fast.ai/
You could also try to reach out to Jeremy or Rachel, see if they can help with anything. They sometimes hang around here on HN too.
AFAIK they promote their Pytorch wrapper in their course instead of using pure Pytorch. Anything else?
In your pricing, you say you are selling, for example, P106 at 0.095$/h, but in your explanation, you are saying you are using idle cycle to mine cryptos (or the reverse, idle cycle to process ML tasks). When I rent a P106, do I have full access to the cores or just partial?
[1] https://serpapi.com
If you want to deploy large-scale computation and significantly reduce your costs, we can help by running mining at the same time under your consent. This only applies to Deep Learning inference.
I understand that you figured out the technical side of things but curious about the human nature side of things. Let's say currently you can have both NN and mining running. From a cryptocurrency price point of view two things can happen:
a. Prices go up - Wont the incentive change towards mining? How do you guys plan to handle such situations?
b. Prices go down - In this case, the intuitive thing to happen is that GPU pricing to get cheaper. But then, will you guys passing on such benefits to your customers? Because the whole point is to be able to make a steady income from the GPUs, mining or not.
If the price goes down, customers will be able to set lower price. As long as GPU holders profit margin is high enough given electricity and maintenance costs, they will do the compute.
If the marketplace matures, pricing of mining, deep learning, rendering and other tasks will be driven by the market. At Snark AI we are working on towards creating this marketplace that will provide optimal benefits to all parties.
I am confused. Let's say your price is $x while public cloud is $x+y. When cryptocurrency price rise the incentive is to increase your price to $x+y, if that is the profit point. In which case you will no longer be cheaper than public cloud. Any differences will be negligible. So, I am not sure you can claim to be always cheaper than public cloud.
You are right my claim that y'<y is slightly weak (was based on "gap is not negligible" assumption, see below).
"gap is not negligible" - means if y' gets near to y, then y will get even higher and there will be always a market gap, which I think you disagree with.
Based on your suggestion, on extreme scenario I would soften my claim to y'=<y, without us making profit. :)