Cheapest Source of x86 Cores?
I'm doing embarrassingly parallel simulations (think Monte Carlo runs of a legacy scientific binary) and am trying to find the cheapest possible host source of x86 compute, at scale. These are jobs that are single-threaded, use maybe 2-4GB of ram, last an hour, and can be checkpointed if necessary. A c5.18xlarge on AWS has 36 physical (real) cores and on the spot market is $0.74/hr which works out to $0.02/core-hour. Does anyone know of cheaper options?
57 comments
[ 3.3 ms ] story [ 119 ms ] threadE.g. GCP C3 pricing is $0.003/vCPU hour and $0.004 for 1GB RAM -- where C3 = Intel Sapphire Rapids (latest gen), which should be quite a bit faster than a C5 (older Intel).
c3-highcpu-88 (44 real cores) is $0.344/hr hence
N2D (AMD Milan) has equivalents that's also in a similar price bracket
See: https://cloud.google.com/compute/vm-instance-pricing#general...
Azure has similar prices for spot with AMD CPU.
On a "smaller" cloud, Oracle cloud E4 (AMD Milan) prices are also a lot cheaper than AWS.
For an even smaller cloud there's also Hetzner cloud with dedicated cores or dedicated servers even.
https://github.com/rcarmo/azure-stable-diffusion#cheapest-sp...
That being said, Hetzner sometimes has weeks where there's no setup fee on some of their servers. Doesn't seem to be the case for these ones at the moment though.
8 dedicated 64t/128gb servers for 2 years, 0 cumulative seconds of downtime, perfect record.
There's something to be said about companies where things work so you don't need support. Some companies make you go through support for IPMI, and sometimes it's self-service and you only need to go through self-service if it breaks.
Even a dedicated server from OVH with similar specs is $0.02-0.04/real-not-smt-core-hour.
Disclosure: I'm an investor and advisor.
Perhaps you made a bad call with this one?
The other commenters are social media randos, who actually don't know WTF they're talking about. Lookup my profile and you can decide if I do.
Sometimes HN are just a herd of lazy jerks.
"The power of 100,000 computers in a few clicks"
No strong impression either way from that. So, lets look a bit more...
k, there's an "About" button up the top. Clicked that.
Nothing. It just drops down a list of "Blog", "Team", or "Contact".
I don't give a shit about any of those, nor have any interest in them.
Why isn't "About" actually taking me to a page with info telling me WTF it's About?
That's not super shady anyway, just really dumb design.
Moving on, Lets look at the "Developers" options. So I click "Developers" then pick the 1st option "Desktop apps". That open's a new submenu, so I pick the first page there... "Dashboard".
Instead of anything useful, that takes me to a website where I need to login.
Well. That's the end of my interest. Closed website, never to return.
At least it doesn't seem shady, as it never took me to anything other than the front page even though I tried (briefly).
You might want to advise them a bit harder or differently or something, as it's clearly not great currently. :/
At this stage of the company, the goal of the website is to provide a validating presence for people who already heard about them, because they're selling to carefully vetted partners.
The typical flow is (strong reason to engage) => homepage => "deploy on Massive" => book a time to talk.
Massive never seems short of customer interest, and the challenge is more on engineering to safely and efficiency grow.
I guess now is the time to plug the jobs page: https://www.joinmassive.com/jobs (I'm personally leading the key searches and feedback very welcome on those listings)
You've just wasted my time, and other people people's time on this crap.
Trying to claim that, after quite literally coming here on HN and trying to spruik that crowd to everyone, just makes the case for the earlier critical commenters.
That's the behaviour of someone clearly full of shit.
Thanks for confirming joinmassive is actually a shady operation.
Hopefully this gets into search engine results, so less people waste time on this bullshit.
Many startups sell and work closely with partners earlier in their lifecycle before their platforms are ready for the mainstream. OP was asking about access to lots of low cost CPUs for an application that could be a good fit, so I posted a casual one-liner. Massive is akin to SETI@Home but with a slightly more general SDK, and which complies with privacy, security and opt-in requirements from the major AV companies.
I personally think it's awesome to have another way to monetize that isn't ads or subscriptions, both of which have downsides to users and don't fit every type of application.
You don't have to agree, but I'd *kindly* ask to be afforded the same respect I'd give you, and which frankly I personally deserve.
None at all.
>Massive is like Airbnb or Turo for your computer. Rather than letting you share your home or vehicle when you’re not using them, Massive lets you share any unused computing resources you have. In exchange, app developers give you access to their premium features.
This doesn't have security concerns, not at all, nope
>Massive combines the small amounts of resources contributed by users like you into a supercomputer that funds app features by mining cryptocurrency, decentralizing blockchain infrastructure, running scientific simulations, and performing general distributed tasks.
Yea so it's totally just going to be abused to crpytomine 24/7 on your hardware
>Massive doesn’t develop consumer apps but has partnered with many developers who offer to let you upgrade with Massive in their apps. Depending on your geolocation, you’ll get such offers in apps like Boom 3D for Windows and Digg Desktop.
....Yea this is basically malware and botnetting. There is 10000% perverse incentive for those "apps" to trick users into the botnet
If it's on S3 it might still be best to use AWS EC2 in the same region.
Oracle is definitely a good choice with 10TB free tier.
Hetzner cloud likely has cheaper compute than Oracle.
Upcloud has the fastest CPU.
If it's for several hundred hours worth of compute, then dedicated servers can start to look pretty good.
eg: https://www.hetzner.com/dedicated-rootserver/ax161
€119.80/mo, for a 32 (real) core Zen2 EPYC with 128GB ram
They generally tend to have better local disk performance than most cloud servers, but that might be irrelevant for your use case. :)
If you're just doing short bursts of use though, then it's probably not the right option.
Doing that moves the cost up to €165.80 for 64 SMT cores with 256GB ram for a full month.
(Note - I've updated those prices as I previously forgot to set the region to "US" for the Hetzner pricing thing)
--
Lets see if my math is working today. ;)
€165.80 / 64 cores = 2.5906 per core, per month
Using 720 hours a month for our purposes, that's €0.0036 per core hour.
--
Use xe.com to do pricing conversion, that €165.80 is US$179.79.
So, $179.79 / 64 cores = $2.809 per core, per month
Using 720 hours a month for our purposes, that's US$0.0039 per core hour.
So, about a fifth of the price of $.02/core-hour.
Though I don't know how much you lose in scheduling trying to keep it fully utilized 24/7 versus the spot pricing.
You pointed out initially that it's 128GB ram for (only) 32 real cores. That's still 4GB ram per core.
What am I misunderstanding? :)
--
That being said, if they're just doing bursty work and won't use servers for long stretches then it's probably not the right choice.
You could also just start a small project to pay people. If 2¢ per core hour is a reasonable price, that means a Ryzen 5950X user could make $7.68 a day not using SMT threads when the machine is mostly idle.
I used to buy dell outlet servers (which is their refurbed gear) all the time for folks who insist on dell, they end up being about 20-50% of the cost of cloud pricing if you are utilizing it with a full workload. Their stuff also includes a 3 year warranty with next day support, so it isn't very risky buying a refurbished server. There have been times where I found deals where the components could be parted out on ebay for more than the cost of the server. If you don't need support, there are tons of used servers on ebay / craigslist where you can get deals on 1-3 year old hardware that are good enough to purchase multiple spares.
For example, you can purchase a 2x 32 core server [1] for ~11k from the outlet store with next day support.
If you are running 24/7, with 64 cores over a 3 year period...
If you wanted to go supermicro [2] with somewhat less reliable support, its a similar story. It would be a 1u dual 96 core (192 cores) box w/ 1.5tb memory for around ~30k. You can recoup your hardware costs within the first year vs. running in AWS if your workloads are high enough to justify the hardware expense.[1] - https://outlet.us.dell.com/ARBOnlineSales/Online/SecondaryIn...
[2] - https://www.itcreations.com/configurator/model/supermicro-as...