With AI making it possible to use natural language to modify code, bare metal can make things easier to use with your own code and customization. Abstractions tend to be harder to reason about and have more limited functionality in exchange for being easier to get started on some standard setup.
For smaller operations I’d still go with a rent-a-server model with AWS. Theirs is a critical mass though where roll your own makes sense.
The long term app model on the market model is shifting much more towards buying services vs renting infrastructure. It’s here where the AWS case falls apart with folks now buying Planet Scale vs RDS, buying DataBricks over the mess that AWS puts for for data lakes, working with model providers directly vs the headaches of Bedrock. The real long term threat is AWS continues to whiff on all the other stuff and gets reduced to a boring rent-a-server shop that market forces will drive to be very low margin.
Yes a lot of those 3rd party services will run on AWS but the future looks like folks renting servers from AWS at 7% gross margin and selling their value-add service on top at 60% gross margin.
What people forget about the OVH or Hetzner comparison is that for those entry servers they are known for, think the Advance line with OVH or AX with Hetzner. Those boxes come with some drawbacks.
The OVH Advance line for example comes without ECC memory, in a server, that might host databases. It's a disaster waiting to happen. There is no option to add ECC memory with the Advance line, so you have to use Scale or High Grade servers, which are far from "affordable".
Hetzner per default comes with a single PSU, a single uplink. Yes, if nothing happens this is probably fine, but if you need a reliable private network or 10G this will cost extra.
I'm so surprised there is so much pushback against this.. AWS is extremely expensive. The use cases for setting up your system or service entirely in AWS are more rare than people seem to realise. Maybe I'm just the old man screaming at cloud (no pun intended) but when did people forget how to run a baremetal server ?
> We have 730+ days with 99.993% measured availability and we also escaped AWS region wide downtime that happened a week ago.
This is a very nice brag. Given they are using their ddos protection ingress via CloudFlare there is that dependancy, but in that case I can 100% agree than DNS and ingress can absolutely be a full time job. Running some microservices and a database absolutely is not. If your teams are constantly monitoring and adjusting them such as scaling, then the problem is the design. Not the hosting.
Unless you're a small company serving up billions of heavy requests an hour, I would put money on the bet AWS is overcharging you.
> I'm so surprised there is so much pushback against this..
Same, this trend towards "AWS all the things" has really amazed me.
We've all mocked small companies copying big companies by trying to make their app super-duper scalable from the very start. After all, everyone things they are the next google, despite their 5 total users right now.
But this is really the opposite. AWS is phenomenal for the startup that would readily trade high opex for lower capex. Servers aren't the cheapest things in the world to buy and they depreciate. It makes total sense for startups to start this way.
But why are big companies, with an actual budget for staff, copying the behavior of their favorite startups?
How would you do multi-region deployments with your own DC?
This is an issue for several companies that start small and within 5 years they find the need to expand abroad. Be it for data sovereignty or so, which is becoming more important than ever in the last 10 years.
Duplicating a region is "a few clicks away" on AWS. This is what the provider enables you to do.
This and a lot of other things. And for such things, yes, you gotta pay.
I am not - I hate AWS(and cloud in general) with a passion - overpriced, you are getting locked in by a closed ecosystem the moment you say "hey this feature is neat it will save me so much work", only to realize that you are stuck paying for it for years if you decide to move away from it. But people are inclined to jump on a hype train and become evangelists for life. Truth is AWS(or GCP or Azure or anything else) is a viable option in two cases:
1. You are making a product with 3 friends on evenings and you want to ship asap without having the capacity to invest and setup infrastructure.
2. You are a huge corporation with tens of thousands of employees and hardware needs that you simply cannot source yourself easily or sort out the collocation of the hardware.
Everyone else - get a dozen second-hand servers, shove them in a rack in a data center and you will own the hardware and everything associated with it at half the price of what you'd be paying AWS in a year.
> when did people forget how to run a baremetal server ?
I don't think people have forgotten, but I think Amazon has done an amazingly excellent job of marketing and developer relations over the years, to the point that they've convinced most developers that doing your own thing is 1) a lot of expensive, specialized work, and 2) actively dangerous for your business, whether it's because of security, uptime, or some other ops bogeyman of the day.
Note that I said "developers". Most developers are not sysadmins or IT operations people. Most of them have never set up Linux on a desktop or laptop, let alone on real server hardware (that they've also set up themselves). Most of them never had the chance to forget how to run a bare-metal server; they never knew how in the first place. (Hell, I've been running desktop Linux for 25+ years, and I don't think I've ever set up Linux on actual server hardware. Closest I've come is bare-metal Solaris, but that was like 25 years ago.)
"DevOps" today usually means that you know how to run a CLI tool or drive a web interface to deploy your automatically-built container artifact to some cloud-based production system that someone else manages, hiding the details from you. (This bit also can be true for shops that run on bare metal, depending on how advanced their own sysadmin/ops team is.) While developers are often not decision-makers in a larger org, they can be at smaller orgs, and once those developers get on the cloud, you probably will stay on the cloud (companies like OneUptime are the exception, not the rule), even if you've gotten much larger and it's stupid expensive to continue running that way.
> when did people forget how to run a baremetal server ?
My opinion on this: docker sort of changed the game here. It sort of enabled a lot of people to get a "new and fresh" level of abstraction to not bother about bare metal.
As an example, I work in company where most consultants are doing DevOps and k8 is big part of that.
What made me consider that? I've been told multiple times that "you know your stuff" when I mention some kernel or userland feature that container approach provides.
In the early days of cloud service providers, they offered a handful of high-value services, all at great prices, making them cost-competitive with bare metal but much easier. That was then.
Things today are different. As cloud service providers have grown to become dominant, they now offer a vast, complicated tangle of services, microservices, control panels, etc., at prices that can spiral out of control if you are not constantly on top of them, making bare metal cheaper for many use cases.
This is the right take - there is a huge variation in "value per dollar" across AWS services. The base ones that solve hard problems like durable persistent state can be very much worth it. They tend to be the older ones.
Microk8s has common, catastrophic performance bugs. There are also catastrophic problems with microk8s Ceph addons. So is this post true? Microk8s, for people who know stuff, is a canary for clusters / applications that don’t really work.
> Our workload is 24/7 steady. We were already at >90% reservation coverage; there was no idle burst capacity to “right size” away. If we had the kind of bursty compute profile many commenters referenced, the choice would be different.
Which TBH applies to many, many places, even if they are not aware of it.
Even if you have that you'll find AWS is "out of stock" and wants you to create reservations that essentially cost the same as just having the machine 24/7.
Nah. They could have just overprovisioned to hell for much cheaper. Boxes at Hetzner cost up to 10 times less than equal level of AWS compute. Just overprovision for cheaper. You have to overprovision on the cloud anyway - you cant risk your users waiting 1-2 minutes until your new nodes/pods come up. So 'cloud is good for spiky load' argument is just a lie we tell ourselves.
> Equinix Metal got the closest, but bare metal on-demand still carried a 25-30% premium over our CapEx plan. Their global footprint is tempting; we may still use them for short-lived expansion.
> The Equinix Metal service will be sunset on June 30, 2026.
Several years off AWS, the only thing I still prefer AWS for is SES, otherwise Cloudflare has the more cost effective managed services. For everything else we use Hetzner US Cloud VMs for hosting all App Servers and Server Software.
Our .NET Apps are still deployed as Docker Compose Apps which we use GitHub Actions and Kamal [1] to deploy. Most Apps use SQLite + Litestream with real-time replication to R2, but have switched to a local PostgreSQL for our Latest App with regular backups to R2.
Thanks to AI that can walk you through any hurdle and create whatever deployment, backup and automation scripts you need, it's never been easier to self-host.
> Cloud makes sense when elasticity matters; bare metal wins when baseload dominates.
This really is the crux of the matter in my opinion, at least for applications (databases and so on is in my opinion more nuanced). I've only worked at one place where using cloud functions made sense (keeping it somewhat vague here): data ingestion from stations that could be EXTREMELY bursty. Usually we got data from the stations at roughly midnight every day, nothing a regular server couldn't handle, but occasionally a station would come back online after weeks or new stations got connected etc which produced incredible load for a very short amount of time when we fetched, parsed and handled each packet. Instead of queuing things for ages we could instead just horizontally scale it out to handle the pressure.
They were running for a long time (months? over a year?) on a single rack in a single datacenter. Eventually they scaled out but the word is eventually. I think that summarizes both sides of this debate in a nutshell. You can move off of AWS but unless you invest a lot you will take on increased risk. Maybe you'll get lucky and your one rack won't burn down. Maybe you won't. They did get lucky.
Maybe look at R2 or Wasabi instead of S3. That would cut your storage bill by 3x and take your cloud network bill to zero. IMO self-managing DBs always sucks no matter what you do.
>We're now moving to Talos. We PXE boot with Tinkerbell, image with Talos, manage configs through Flux and Terraform, and run conformance suites before each Kubernetes upgrade.
Gee, how hard is to find SE experts in that particular combination of available ops tools? While in AWS every AWS certified engineer would speak the same language, the DIY approach surely suffers from the lack of "one way" to do things. Change Flux with Argo for example (assuming the post is talking about that Flex and no another tool with the same name), and you have a almost completely different gitops workflow. How do they manage to settle with a specific set of tools?
> Gee, how hard is to find SE experts in that particular combination of available ops tools?
You find expert in Ops, not in tools. People that know the fundamentals, not just the buttons to push in "certain situations" without knowing what's really going on under the hood.
AWS is extremely expensive, and I think I have to agree with DHH's assessment that many developers are afraid of computers. AWS is taking advantage of that fear of actually just setting up linux and configuring a computer.
However to steelman AWS use. Many businesses are STILL running mainframes. Many run terrible setups like Access as a production database. In 2025 there are large companies with no CICD platforms or IAC, and some companies where even VC is still a new concept or a dark art. So not every company is in the position to actually hire competent system administrators and system engineers to set up some bare metal machines and configure Ceph, much less Hadoop or Kubernetes. So AWS lets these companies just buy this capabilities while forcing the software stack to modernize.
This is a completely meaningless article if they don't provide information about their technical stack, which AWS services they used to use, what TPS they are hitting, what storage size they're using, etc.
The story will be different for every business because every business has different needs.
Given the answer to "How much did migration and ongoing ops really cost?" it seems like they had an incredibly simple infrastructure on AWS, and it was really easy to move out. If you use a wider-range of services the cost savings are much more likely to cancel themselves.
Small team in a large company who has an enterprise agreement (discount) with a cloud provider? The cloud can be very empowering, in that teams who own their infra in the cloud can make changes that benefit the product in a fraction of the time it would take to work those changes through the org on prem. This depends on having a team that has enough of an understanding of database, network and systems administration to own their infrastructure. If you have more than one team like this, it also pays to have a central cloud enablement team who provides common config and controls to make sure teams have room to work without accidentally overrunning a budget or creating a potential security vulnerability.
Startup who wants to be able to scale? You can start in the cloud without tying yourself to the cloud or a provider if you are really careful. Or, at least design your system architecture in such a way that you can migrate in the future if/when it makes sense.
This is a tech company and it’s adjacent to their core competency. Most companies wouldn’t know MicroK8s from a brand of cereal, they’d only create a mess if they tried this themselves.
Never heard of Talos before now. That looks pretty cool and I might start playing with that on my home lab. Can't use it at work for reasons, but good to keep on top of tech (even if I am a little behind)
91 comments
[ 2.6 ms ] story [ 91.4 ms ] threadThe long term app model on the market model is shifting much more towards buying services vs renting infrastructure. It’s here where the AWS case falls apart with folks now buying Planet Scale vs RDS, buying DataBricks over the mess that AWS puts for for data lakes, working with model providers directly vs the headaches of Bedrock. The real long term threat is AWS continues to whiff on all the other stuff and gets reduced to a boring rent-a-server shop that market forces will drive to be very low margin.
Yes a lot of those 3rd party services will run on AWS but the future looks like folks renting servers from AWS at 7% gross margin and selling their value-add service on top at 60% gross margin.
What people forget about the OVH or Hetzner comparison is that for those entry servers they are known for, think the Advance line with OVH or AX with Hetzner. Those boxes come with some drawbacks.
The OVH Advance line for example comes without ECC memory, in a server, that might host databases. It's a disaster waiting to happen. There is no option to add ECC memory with the Advance line, so you have to use Scale or High Grade servers, which are far from "affordable".
Hetzner per default comes with a single PSU, a single uplink. Yes, if nothing happens this is probably fine, but if you need a reliable private network or 10G this will cost extra.
> We have 730+ days with 99.993% measured availability and we also escaped AWS region wide downtime that happened a week ago.
This is a very nice brag. Given they are using their ddos protection ingress via CloudFlare there is that dependancy, but in that case I can 100% agree than DNS and ingress can absolutely be a full time job. Running some microservices and a database absolutely is not. If your teams are constantly monitoring and adjusting them such as scaling, then the problem is the design. Not the hosting.
Unless you're a small company serving up billions of heavy requests an hour, I would put money on the bet AWS is overcharging you.
Same, this trend towards "AWS all the things" has really amazed me.
We've all mocked small companies copying big companies by trying to make their app super-duper scalable from the very start. After all, everyone things they are the next google, despite their 5 total users right now.
But this is really the opposite. AWS is phenomenal for the startup that would readily trade high opex for lower capex. Servers aren't the cheapest things in the world to buy and they depreciate. It makes total sense for startups to start this way.
But why are big companies, with an actual budget for staff, copying the behavior of their favorite startups?
This is an issue for several companies that start small and within 5 years they find the need to expand abroad. Be it for data sovereignty or so, which is becoming more important than ever in the last 10 years.
Duplicating a region is "a few clicks away" on AWS. This is what the provider enables you to do.
This and a lot of other things. And for such things, yes, you gotta pay.
1. You are making a product with 3 friends on evenings and you want to ship asap without having the capacity to invest and setup infrastructure. 2. You are a huge corporation with tens of thousands of employees and hardware needs that you simply cannot source yourself easily or sort out the collocation of the hardware.
Everyone else - get a dozen second-hand servers, shove them in a rack in a data center and you will own the hardware and everything associated with it at half the price of what you'd be paying AWS in a year.
I don't think people have forgotten, but I think Amazon has done an amazingly excellent job of marketing and developer relations over the years, to the point that they've convinced most developers that doing your own thing is 1) a lot of expensive, specialized work, and 2) actively dangerous for your business, whether it's because of security, uptime, or some other ops bogeyman of the day.
Note that I said "developers". Most developers are not sysadmins or IT operations people. Most of them have never set up Linux on a desktop or laptop, let alone on real server hardware (that they've also set up themselves). Most of them never had the chance to forget how to run a bare-metal server; they never knew how in the first place. (Hell, I've been running desktop Linux for 25+ years, and I don't think I've ever set up Linux on actual server hardware. Closest I've come is bare-metal Solaris, but that was like 25 years ago.)
"DevOps" today usually means that you know how to run a CLI tool or drive a web interface to deploy your automatically-built container artifact to some cloud-based production system that someone else manages, hiding the details from you. (This bit also can be true for shops that run on bare metal, depending on how advanced their own sysadmin/ops team is.) While developers are often not decision-makers in a larger org, they can be at smaller orgs, and once those developers get on the cloud, you probably will stay on the cloud (companies like OneUptime are the exception, not the rule), even if you've gotten much larger and it's stupid expensive to continue running that way.
People simply don't believe bare metal is better because of this conditioning towards the cloud.
Every company should strive for self-sufficiency, but this idea is not that widespread in the software industry.
My opinion on this: docker sort of changed the game here. It sort of enabled a lot of people to get a "new and fresh" level of abstraction to not bother about bare metal.
As an example, I work in company where most consultants are doing DevOps and k8 is big part of that.
What made me consider that? I've been told multiple times that "you know your stuff" when I mention some kernel or userland feature that container approach provides.
Things today are different. As cloud service providers have grown to become dominant, they now offer a vast, complicated tangle of services, microservices, control panels, etc., at prices that can spiral out of control if you are not constantly on top of them, making bare metal cheaper for many use cases.
It’s like dude if you think AWS is raw, sushi restaurants must really confuse you.
So yeah in a hot economy anything you launch grows crazy. Then you do the thought leader talk circuit.
And in a cold economy, you can stop growing, optimize for your 24/7 steady workload, and also do the thought leader talk circuit.
> The Equinix Metal service will be sunset on June 30, 2026.
https://docs.equinix.com/metal/
Our .NET Apps are still deployed as Docker Compose Apps which we use GitHub Actions and Kamal [1] to deploy. Most Apps use SQLite + Litestream with real-time replication to R2, but have switched to a local PostgreSQL for our Latest App with regular backups to R2.
Thanks to AI that can walk you through any hurdle and create whatever deployment, backup and automation scripts you need, it's never been easier to self-host.
[1] https://docs.servicestack.net/kamal-deploy
This really is the crux of the matter in my opinion, at least for applications (databases and so on is in my opinion more nuanced). I've only worked at one place where using cloud functions made sense (keeping it somewhat vague here): data ingestion from stations that could be EXTREMELY bursty. Usually we got data from the stations at roughly midnight every day, nothing a regular server couldn't handle, but occasionally a station would come back online after weeks or new stations got connected etc which produced incredible load for a very short amount of time when we fetched, parsed and handled each packet. Instead of queuing things for ages we could instead just horizontally scale it out to handle the pressure.
Is there a simple safe setup that we can run on an Ubuntu server?
We self-host the Postgres db with frequent backups to s3 but just in case the site takes off, we need an affordable reliable solution.
Does anyone here run their own db servers? Any advise?
Backups, security, upgrades etc
Gee, how hard is to find SE experts in that particular combination of available ops tools? While in AWS every AWS certified engineer would speak the same language, the DIY approach surely suffers from the lack of "one way" to do things. Change Flux with Argo for example (assuming the post is talking about that Flex and no another tool with the same name), and you have a almost completely different gitops workflow. How do they manage to settle with a specific set of tools?
You find expert in Ops, not in tools. People that know the fundamentals, not just the buttons to push in "certain situations" without knowing what's really going on under the hood.
However to steelman AWS use. Many businesses are STILL running mainframes. Many run terrible setups like Access as a production database. In 2025 there are large companies with no CICD platforms or IAC, and some companies where even VC is still a new concept or a dark art. So not every company is in the position to actually hire competent system administrators and system engineers to set up some bare metal machines and configure Ceph, much less Hadoop or Kubernetes. So AWS lets these companies just buy this capabilities while forcing the software stack to modernize.
The story will be different for every business because every business has different needs.
Given the answer to "How much did migration and ongoing ops really cost?" it seems like they had an incredibly simple infrastructure on AWS, and it was really easy to move out. If you use a wider-range of services the cost savings are much more likely to cancel themselves.
Very much this.
Small team in a large company who has an enterprise agreement (discount) with a cloud provider? The cloud can be very empowering, in that teams who own their infra in the cloud can make changes that benefit the product in a fraction of the time it would take to work those changes through the org on prem. This depends on having a team that has enough of an understanding of database, network and systems administration to own their infrastructure. If you have more than one team like this, it also pays to have a central cloud enablement team who provides common config and controls to make sure teams have room to work without accidentally overrunning a budget or creating a potential security vulnerability.
Startup who wants to be able to scale? You can start in the cloud without tying yourself to the cloud or a provider if you are really careful. Or, at least design your system architecture in such a way that you can migrate in the future if/when it makes sense.