I think it's incentives. Working with AWS is good for your resume. It's also a responsibility thing. If AWS is down then it's Amazon's fault. If there's a problem with your physical server, then it's your problem.
A broader thing here is -- and you may also notice this trend in software -- employees are incentivized towards complex solutions, while business owners are incentivized towards simple solutions.
"Unfortunately, the [Hetzner] CPX22is available only in eu-central and ap-southeast, but if that’s OK with you it is the best value and fastest overall."
I am still running ROME epyc cpus that I picked up for couple hundred and they're doing great. Power usage is not the best and singlethread is awful (-50%), but multithreaded performance kicks 9950x in the ass at around 90k vs 70k.
Anyone have experience with Oracle Cloud and ease of moving away?
This benchmark seems to recommend Oracle Cloud, but I’ve heard that Oracle has historically used aggressive licenses and legal terms to keep customers locked-in.
GCP (near the top) 3 years reserved 16 ARM cores + 120 GB costs + 1TB local SSD costs > $1k/month. It does not even match the specs of a Ryzen AI 395 Framework miniPC that goes for ~$4k AFAIK.
You can extract a lot of value from bare-metal servers from Hetzner but you need to put some effort initially to get them going. That being said, it is not really that difficult. And, frankly, it a lot more fun.
You can't compare VPS with VMs from major cloud provider, VPS don't offer anything beside basic compute.
Also virtualization from cloud provider is way better because they have custom hardware and software so you don't suffer from noisy neighbours for example.
I just ran some massive tests on our own CI. I use AMD Turin for this on gcp, which was noted as one of the fastest ones in the article.
The most insane part here is that the AMD EPYC 4565p can beat the turin's used on the cloud providers, by as much as 2x in the single core.
Our tests took 2 minutes on GCP, 1 minute flat on the 4565p with its boost to 5.1ghz holding steady vs only 4.1ghz on the gcp ones.
GCP charges $130 a month for 8vcpus. ALSO this is for SPOT that can be killed at any moment.
My 4565p is a $500 cpu... 32 vcpus... racked in a datacenter. The machine cost under 2k.
i am trying hard to convince more people to rack themselves especially for CI actions. The cloud provider charging $130 / mo for 3x less vcpus you break even in a couple months, it doesn't matter if it dies a few months later. On top of that you're getting full dedicated and 2x the perf. Anyways... glad to see I chose the right cpu type for gcloud even though nothing comes close to the cost / perf of self racking
A desktop class CPU is definitely quicker single threaded and multithreaded, no surprises there most of these are dual core. The single threaded performance of the C8A is actually pretty good but its also the best of the bunch by a wide margin most of the CPUs are far behind. Memory performance appears to be attrocious all around.
Disclosure: I work on VMs at Google Compute Engine :)
This was a really, really good write-up. I appreciated the breadth of VMs tested and the spread of benchmarks. A few random observations:
1. Turin is a beast.
2. The data on price-performance makes Hetzner look really fantastic, especially for small scale projects where region placement doesn’t matter much and big bursty scaling isn’t required.
Needs: network performance and costs, desperately.
I wouldn't mind SAN/non-local storage performance and costs too.
VM cost isn't where AWS gets you. It's ALLLLLLLL the other nickel and diming that they kill you on, especially since outbound data transfer costs a log of money to get off of the platform.
Awesome write up. I use EC2 t4g instances extensively as I tend to scale workloads horizontal (using Kubernetes, Docker) and admittedly the workloads aren’t cpu constrained. Would be interesting to see how t4g’s compare to other low end comparable VMs on GCP, DigitalOcean, Hetzner. Though I get the difficulty testing as it requires maxing out CPU credits (waiting) for accurate results before starting testing for each VM.
Looking forward to t5g’s whenever (if ever) they release.
There are definitely ways to get some really good hardware from lowend shops as well.
Hetzner is pretty decent too actually but I actually think that OVH might be even cheaper while still being competitive, but in the case of OVH, I think one minor issue people can have is the setup fee at times but time to time like during black fridays or special deals, there are ways to get 0 setup fees.
If someone wants something stable, I think that OVH is pretty great as well and is comparable to Hetzner in terms of pure price but I have heard that given their scale, their support can be 50/50 but I recommend joining their discord and maybe even using (twitter oof) to message them as thsi was something which worked for someone on hackernews the last time something like this was discussed.
If you are okay with some more steal factor, use netcup.
You should also probably look at gaming type servers too. I know a person who is a one man shop who was more passionate about these stuff and did have high-end hardware (200k$) in investment in his provider.
Going more into the specifics of finance from provider side, usually the idea is that they recoup the costs in 5 years if running at decent capacity/having sold quite a bit but the first few years so as in example of the 200k$, they currently make IIRC 40k-60k$ per year, you have to somehow find your way to customers but that is being messed up because of AI ram prices as their costs to fix any broken hardware has absolutely skyrocketed eating much of their profits.
It's an extremely competitive market at times so if you are looking for something specific. You can actually ask about it in forums like lowendtalk, lowendspirit and these people/providers can respond matching what you are looking for price/performance in for and they can also provide test servers if need be and there is a unified form of benchmarking within this space called yabs (yet-another-benchmark-script) which you can ask for a provider.
But in all regards, if someone doesn't want to go through this hassle, I will just say that the same provider that I mentioned earlier had also in public mentioned how hetzner prices are pretty competitive and in all honesty I agree with that too.
If someone doesn't want to go through some/any part of this hassle, then hetzner/OVH are my go-to safe options. They are big enough that their downtime shouldn't be blamed on you for picking them while being really good in themselves. (Something which I have actually heard quite often mentioned on hackernews)
And within this space, if you don't absolutely know what you want, there is no generic winner as there are niches that are occupied. So if you ever want to find any alternative to hetzner for even more cheaper, its best that you first decide what are the things exactly that you want and then ask rather than see for pre-existing if possible.
Also if possible, make deals during blackfriday/cyber-monday. Not sure about Hetzner but in OVH/other providers, you can get recurring deals and the sever setup costs etc. removed sometimes.
Vultr topped the benchies a few years ago. Not sure how they fare now, would be good to see some more recent benchmarks. Was sad to see them missing from this one.
Would have been nice to see Scaleway included, I recently migrated from Digital Ocean to them and found them to be very similar in pricing and performance.
32 comments
[ 2.7 ms ] story [ 44.7 ms ] threadEvery big corporate I have worked at has lower cost of capital than Amazon, and yet they want to move to AWS. I just dont understand it.
A broader thing here is -- and you may also notice this trend in software -- employees are incentivized towards complex solutions, while business owners are incentivized towards simple solutions.
(Shiny object syndrome sold separately ;)
This benchmark seems to recommend Oracle Cloud, but I’ve heard that Oracle has historically used aggressive licenses and legal terms to keep customers locked-in.
Also virtualization from cloud provider is way better because they have custom hardware and software so you don't suffer from noisy neighbours for example.
The most insane part here is that the AMD EPYC 4565p can beat the turin's used on the cloud providers, by as much as 2x in the single core.
Our tests took 2 minutes on GCP, 1 minute flat on the 4565p with its boost to 5.1ghz holding steady vs only 4.1ghz on the gcp ones.
GCP charges $130 a month for 8vcpus. ALSO this is for SPOT that can be killed at any moment.
My 4565p is a $500 cpu... 32 vcpus... racked in a datacenter. The machine cost under 2k.
i am trying hard to convince more people to rack themselves especially for CI actions. The cloud provider charging $130 / mo for 3x less vcpus you break even in a couple months, it doesn't matter if it dies a few months later. On top of that you're getting full dedicated and 2x the perf. Anyways... glad to see I chose the right cpu type for gcloud even though nothing comes close to the cost / perf of self racking
That is ... hard to believe for a CPU-bound task. Do you have any open benchmark which can reproduce that?
7 Zip benchmark
9800X3D 130 GIPs compression, 134 GIPs decompress.
C8A 21577 MIPs (21.5GIPs) compression, 9868 MIPS decompression (9.9GIPs).
Geekbench 5
9800X3D 16975 multithread, 2474 single thread
C8A 4049 multithread, 2240 single thread
A desktop class CPU is definitely quicker single threaded and multithreaded, no surprises there most of these are dual core. The single threaded performance of the C8A is actually pretty good but its also the best of the bunch by a wide margin most of the CPUs are far behind. Memory performance appears to be attrocious all around.
This was a really, really good write-up. I appreciated the breadth of VMs tested and the spread of benchmarks. A few random observations:
1. Turin is a beast.
2. The data on price-performance makes Hetzner look really fantastic, especially for small scale projects where region placement doesn’t matter much and big bursty scaling isn’t required.
3. I think the first ever cloud VM I ever provisioned was on DigitalOcean. I was surprised at how old their fleet was, but I guess they have some limited Emerald Rapids offerings now: https://www.digitalocean.com/blog/introducing-5th-gen-xeon-p...
I wouldn't mind SAN/non-local storage performance and costs too.
VM cost isn't where AWS gets you. It's ALLLLLLLL the other nickel and diming that they kill you on, especially since outbound data transfer costs a log of money to get off of the platform.
Looking forward to t5g’s whenever (if ever) they release.
Hetzner is pretty decent too actually but I actually think that OVH might be even cheaper while still being competitive, but in the case of OVH, I think one minor issue people can have is the setup fee at times but time to time like during black fridays or special deals, there are ways to get 0 setup fees.
If someone wants something stable, I think that OVH is pretty great as well and is comparable to Hetzner in terms of pure price but I have heard that given their scale, their support can be 50/50 but I recommend joining their discord and maybe even using (twitter oof) to message them as thsi was something which worked for someone on hackernews the last time something like this was discussed.
If you are okay with some more steal factor, use netcup.
You should also probably look at gaming type servers too. I know a person who is a one man shop who was more passionate about these stuff and did have high-end hardware (200k$) in investment in his provider.
Going more into the specifics of finance from provider side, usually the idea is that they recoup the costs in 5 years if running at decent capacity/having sold quite a bit but the first few years so as in example of the 200k$, they currently make IIRC 40k-60k$ per year, you have to somehow find your way to customers but that is being messed up because of AI ram prices as their costs to fix any broken hardware has absolutely skyrocketed eating much of their profits.
It's an extremely competitive market at times so if you are looking for something specific. You can actually ask about it in forums like lowendtalk, lowendspirit and these people/providers can respond matching what you are looking for price/performance in for and they can also provide test servers if need be and there is a unified form of benchmarking within this space called yabs (yet-another-benchmark-script) which you can ask for a provider.
But in all regards, if someone doesn't want to go through this hassle, I will just say that the same provider that I mentioned earlier had also in public mentioned how hetzner prices are pretty competitive and in all honesty I agree with that too.
If someone doesn't want to go through some/any part of this hassle, then hetzner/OVH are my go-to safe options. They are big enough that their downtime shouldn't be blamed on you for picking them while being really good in themselves. (Something which I have actually heard quite often mentioned on hackernews)
And within this space, if you don't absolutely know what you want, there is no generic winner as there are niches that are occupied. So if you ever want to find any alternative to hetzner for even more cheaper, its best that you first decide what are the things exactly that you want and then ask rather than see for pre-existing if possible.
Also if possible, make deals during blackfriday/cyber-monday. Not sure about Hetzner but in OVH/other providers, you can get recurring deals and the sever setup costs etc. removed sometimes.