Ask HN: Cloud vs. Dedicated Hosting
I've been trying to convince my CTO that we would gain more benefits from a dedicated hosting solution (Hetzner) but haven't managed to convince him yet.
Right now we have around 5 small dedicated VMs for each dev (2 cores, 4gb ram) and 2 production VMs (4 cores, 16gb ram).
His arguments are that things are easier to manage in the cloud and he doesn't want to have the extra burden of manually configuring infrastructure since none of us have any real background in systems administration and in the future when we will scale to more production machines it would be easier to just start another VM than wait for a new dedicated server. We also use Azure managed MySql which helps us to see performance easier than if we rolled our own.
Do you agree with me and if so what arguments should I use? Would you recommend something other than Hetzner?
97 comments
[ 3.1 ms ] story [ 196 ms ] threadIf it was my money, yes, dedicated FTW. Egress charges... nien danken.
If it's you boss' money, what's the point arguing it. Just let him have his cloud bills.
Also, while cloud pricing can be daunting, once you reach a point when Azure costs to much, you can look at optimizing things. There are a lot of techniques to keep cloud cost acceptable. Look up "FinOps" if you haven't already.
Your solution is to spend time and money on reducing costs. This only makes sense when the costs to move outweigh the costs to reduce costs.
Don’t argue with him, provide numbers for the perspective he might not see. Tell him you want to help him make the best decision. He might loosen up on budget for cloud if he see that you are fumbling around to save a few Go of RAM.
Even a spreadsheet with such numbers is primarily speculation and your upper management knows this (they make such things all the time). As such, if they did not want to agree with you before this spreadsheet won't make a difference. If they want to agree but don't want to take the time to present exactly that spreadsheet to their compatriots, it is worth making it. If you don't work for a very large company (essentially a bureaucracy), this situation probably will not occur.
There is a line between premature optimization and total resource ignorance.
But it sounds like your boss has the idea cloud is where it's at, it's hard to shift. I'd put numbers on it like how many less connections the cloud he wants to pay for can handle, vs the hosted solution for less $$$.
- You ignore resource usage and move to dedicated hosting. You are now wasting money on inefficient processes. To remedy this you apply software engineering practices to understand the model and design it correctly.
- You ignore server unit costs and stay on cloud, but improve your runtime characteristics. You are now wasting money on the hardware, but your software is efficient. To remedy this you have to learn how to manage infra and take it on as constant maintenance.
It seems on your team it makes more sense to me to just pay for the cloud and stay in your software wheel-house.
Having a self-hosted server eliminates a lot of these questions. For hosting in production a managed cloud VM might be sensible as these services often provide additional functions like doing health checks, automatic renewal for tls certs, protection against attacks, etc..... Sure, you can get that on your personal server too, but it does need maintenance.
For development/testing/staging I prefer to have a non-managed server. That said, for most applications it is enough to keep the system of the host up to date and that is easier today than ever before and you might even be faster here than large cloud providers who have a bit of a momentum until problems get addressed.
Also cloud providers do change their infrastructure and might force you to adapt. That can mean extra work out of the blue. They tend to leave your systems as they are, but if you need to update certain components you have to read up on their docs again.
To me looks like your are trying to solve a non-problem.
I use Hetzner dedicated servers for some non critical project, I find it fun, but it’s time consuming and requires some knowledge.
Just one example, having an encrypted storage on a dedicated Linux server is not trivial.
If you don't trust your cloud provider enough to let them manage your encryption, you should probably not use their services. If you use an cloud provider from USA, you should trust the USA too as they can access your data without letting you know. It probably applies to other countries too, but the big cloud providers are from USA.
However with dedicated servers the next step is more complicated for both the hoster and the potential other attackers, since the only chance they have here is to catch the key material during reboots, which is quite a bit more noisy than silently extracting the key from ram of a running VM (especially since they would need to mirror the disks or other complications to have access at the same time as you are using the machine).
If it's a colocated machine (or if the hoster trusts you enough with their dedicated machines) one can use TPM/Secure Boot to make the MITM attack very very difficult (since one can use something like dropbear to have an encrypted and authenticated connection to enter the encryption credentials and TPM/Secureboot can prevent manipulation of the bootloader).
The theft scenario is exactly the reason why you wouldn't want to trust Azure to do the encryption for you, since they can just extract the plain text data from the virtual machine. Unless you are saying you don't trust Azure datacenter techs only?
And for preventing the leakage from old storage devices very non-fancy and simplistic encryption setups are good enough (most of the complexity comes from trying to prevent/detect MITM or manipulation by the hosting provider).
Yes, and also the local datacenter people. I understood that Microsoft didn’t build a datacenter in every region and rents some space in existing datacenters managed by other companies.
If you're concerned about operational and cost efficiency, keep a thought in the back of your mind that one day your scale would need you to host things yourself. Your design choices should work with you at that point rather than lock you in some proprietary platform. But, the key is to get to the point where that becomes a problem. That day is not today. Azure VMs, AWS something whatever, get to your product features and get yourself to a spot where you need to worry about scaling and the savings it would bring.
Sometimes it does outright, see Heroku, or AWS Lambda.
Sometimes you try to bite more than you feel comfortable to chew, and get bogged down configuring kubernetes or terraform when you don't need as much.
In the simplest case, I think, the cloud is also an inevitability. Can you imagine starting up by buying some used servers, colocating them in some datacenter, calling an operator when they need tweaking a BIOS setting, scheduling a visit and downtime when you need to replace a failing NIC, or a degrading HDD? This is what life without the cloud used to look like. Yes, a EC2 instance is cloud.
Of course you can try a middle ground, dedicated hardware managed by a provider, e.g. what Hetzner offers. If your load is very stable and predictable, your stack is very simple and mature (an RDBMS + Apache + PHP, ore maybe Erlang / Elixir), and you are ready to either keep a hot spare or sustain rare but prolonged downtimes, you'd be fine with a small amount of dedicated hardware.
A large amount of dedicated hardware may be economical, but only at quite large scale (see, well, Facebook).
Also failed PSUs or hard drives don't require downtime, since those are hot-swappable and redundant in servers. If you want to optimize for (physical) hands-off operation you can even add an extra 1-2 hard drives that are configured as hot spare that the RAID (be it hardware, software or ZFS) will automatically use in case one of the active ones fails.
And the performance difference between dedicated hardware (be it dedicated server or your own) can be quite staggering compared to what you actually get from the cloud providers.
As someone who is freelancing (and using Hetzner-cloud + dedicated) and also working as a cloud-ops (using AWS and mainly Azure as cloud providers), I can say that going with a cloud provider like Azure is simpler.
Saving cost is definitely important, however, seeing that there are multiple dev/salaried, I doubt it's really a big concern (you also don't seem to have that many resources).
Most of the time, with resources from Azure, you would get features already completed and existing (I am looking directly at Application Insights for example) - with resources like App services, Web apps and Function apps you get ease of debugging (yes, there is a cost associated, however, in the long term when you scale, you definitely see the advantage).
I've personally suggested my colleagues to try out a dedicated server from either Hetzner or OVH (that specific one needed at least 128GB RAM) so going with Azure would of been quite expensive (and probably less than $200 per month from either OVH or Hetzner) - but they've decided to stay with Azure and I can confirm that they are not paying less than $900 per month.
It seems like cost isn't always the main factor, but rather ease the time it would take to prepare/set up everything.
We're working on a solution to gives you an experience like Heroku on AWS without breaking the bank, and I'm pretty sure we won't be the only one working on such a solution.
Flexibility of cloud and simplicity and convenience of services like what we're building will boost the speed massively, plus you'll never be concerned about outgrowing the solution as everything is built on top of AWS.
Spinning up a new VM takes a few seconds, just like on Azure, and ordering a dedicated server is only a click away if you really need it. So you get the best of both worlds.
I don't believe that you can be "too small" to save on infastructure. Hosting our setup on Google Cloud would probably cost 10x what we currently pay for, maybe more considering that our largest server has 32 physical cores...
[0] Blog article about our setup: https://pirsch.io/blog/techstack/
If you're just running plain Linux VM's running some software without need for additional AWS-like services or fancy orchestration, it's good value for money.
I also have a couple of dedicated Hetzner hosts - also remarkably stable, but it will of course take longer to rebuild if it goes down. You should have a continuity plan in place.
your aws infra: 600-1000 month? hetzner: 200 per month?
Is 800/month gonna make a difference to your organization?
Imagine you launch new feature with performance regressions and suddenly your database is at 100% CPU usage
Managed database service will allow you to scale up the instance in a few clicks, so you can redo the feature with a better plan, instead of rushing to roll back or deploy a hot fix.
Hetzner provides cheap and performant VPSses. Depending on the geo, it might be a better fit.
CTO says they have to expertise in "systems administration", yet they're running VPSes, and not managed services. This is exactly where some dev installs some software which is accessible by default. There you have your little dataleak.
So imo, CTO is wrong, but post doesn't have to be right per se.
In the early stages of product development, how often does this scenario occur where your database is at 100% CPU utilization and the solve is not a new database index but instead a hotfix?
AWS bill is 5k plus per month
One deployment lead to regressions (^2 growth growing along with usage of the new feature)
When it became a problem I just upgraded the instance and made a ticket
the ticket was solved within next sprint, so 2 weeks or so minimum
in these 258 euros I have 4 dedicated servers that never went down so far. One of the 4 is only for the database while the other 3 are for kubernetes which is managed and deployed using Rancher. Out of the 3, my cpu usage is 4 out of 28 so plenty of room to grow as everything is on overkill.
I do daily backups of the database and upload them to an S3. I know this is not the perfect solution but, for a company that small (mine is similar) $61k are not something to be ignored. I used that money on Adwords which brings much more value than spending huge amounts on something which rarely "shines" for my use case. In fact, I had more downtime in the cloud than I had with Hetzner - mostly because they perform updates (to kubernetes, to database engine etc) or they simply have outages.
Everything is subjective.
not the same as VM to dedicated
5 small VMs, 2 big VMs, 1 managed db
For the sake of security and performance you're better off using azure (especially the hosted database).
Otherwise you'll need to configure iptables, backups, software updates, high availability (assuming it's a requirement) all by yourself. If you don't have experience in these things you don't want to be learning how to do them with the thing that generates your revenue.
If you are selling to enterprise / businesses AWS or Microsoft are much better places to be reputation wise I think.
https://aws.amazon.com/compliance/programs/ To get a feel for the paper pushing exercise they’ve spent time on.
BTW - I put a large memory instance on ECS anywhere, worked great - if you have very high memory needs something to consider - a basic dell gets you 3TB these days - very nice for development
I'd challenge the idea of dedicated infrastructure per developer in the first place. There's little reason why this could not be done locally on a developer machine using Vagrant, local Kubernetes, Docker Compose, whatever.
If you have more significant compute needs a dedicated server can start to make sense. Linux admin is becoming a bit of a lost art in this container-everything world but actually it's not that hard. I run a hardware server for a lab and once it's running there's very little work, super reliable, and very cost effective.
https://youtu.be/k3bUvZf89k8?t=697
A lot of it depends on what you're actually doing, but for a B2B SaaS app you could potentially make an argument around vertical scaling. Nowadays you can spin up a 128 vCPU VM with 2TB memory on EC2 in minutes, so why bother having lots of separate smaller servers? I've seen people making this argument quite well recently, especially in relation to using things like SQLite instead of a networked RDBMS. But.. I can't see you making such an argument very convincingly unless you're heavily into ops or play with this sort of stuff on the regular, so I'd stick with your CTO's approach for now and keep alternative ideas in your pocket for hackathons, spare time, or prototyping sessions.
For 3600$ [0] per month. Spending this kind of money makes sense if plan B is to spend man-years optimizing your app or if you have an insanely expensive per-machine license, but otherwise you are better off with multiple servers - not only for price, but especially for availability.
[0] Assuming a c6a.32xlarge in us-east1, with only 256GB of RAM. Dedicated machines are more expensive.
None of your dev team has a background in sys admin or managing a DB? You're going to be in for a period of difficulty while you gain those skills and that's going to be felt quite heavily on a team of 5. Now if only one of you picks up that skillset now you've also got a high Bus Factor (1) in the event something goes wrong with your self-hosted infra.
It's entirely possible that you're right in the long term but in the short term it doesn't make sense because your team is too small and the cost difference, while probably proportionally large, is almost certainly tiny in absolute terms.
I would explore the possibility that you want to be able to easily move off of Azure in the future. This may or may not be worth planning for but it likely makes more sense than going to self-hosted right now. If you can get buy-in that you shouldn't be on Azure long-term then you should avoid Azure specific services that will make it harder to switch. This would mean keeping your stack simple - compute and DB where possible. Avoiding things like Event Grid, their PubSub product and possibly even their CI/deployment offering.
Again - even doing that very possibly does not make business sense and you shouldn't approach that analysis with a specific conclusion in mind. Does your product need something like their AI offerings? If so it's probably a lot faster and more effective to lean into the products they're offering than trying to run an alternative yourself on their compute.
(1) - https://en.wikipedia.org/wiki/Bus_factor
For most B2B businesses offering a SaaS product, I don't think on-demand variable "web scalability" is the decision framework for AWS/Azure/GCP.
Instead, it's really about faster product development iteration. If new SaaS product features can be delivered to market faster because it can leverage many of the higher-level managed services in the AWS/Azure tech stack portfolio like DynamoDB or Cosmos DB -- without your 5 dev team building the equivalent tech stack from scratch, that's when paying the profit margins to AWS/Azure are worth it.
In other words, your CTO and the devs have to look at your future product roadmap and see which features would require extra developer costs in re-inventing aspects of AWS/Azure that may negate the cost savings of a "dumb" IaaS like Hetzner.
If there are actual development costs from moving from Azure to dedicated servers then they've already coupled to Azure and at a very early stage. The point of giving credits paid for for Microsoft. It would also seem they've unknowningly coupled too which should be worrying. I would be very worried about a CTO who unknowningly coupled to anything.
Your CTO is smart.
If you aren't paying the bills use Azure. I do pay the bills for my product and use Hetzner.
Also, you can use vms from other vendors. You don't have to go full dedicated.
You have two small prod servers. Yes, other cloud services would be cheaper but it's probably not worth the migration time. Stepping on dollars to pick up pennies.
Regarding production VMs: How much do you expect to save by moving to self-managing everything? I’d expect a few dollars saved for the hardware, but a huge amount of time spent setting everything up and documenting it properly. Net loss.
Let me tell you, I host a B2B app on Hetzner because I cannot use typical cloud providers due to GDPR, and I hate it. So much fiddling around to get everything working and I still don’t have high availability because I don’t have the expertise in that area. It’s a huge time sink and takes significant time away from actual development.
Conclusion: Use your time and money wisely. Do what speeds up adding value to the business. Saving a few pennies but missing out on growth and recurring revenue is a bad deal.
If you don't have sysadmin that could tackle dedicated hosting but you have knowledge to keep things running in cloud - stick with the cloud for now.
For instance, both my postgres master and replica went down one day the other weekend, because another machine in the same rack popped the breaker. In my specific application, I can cope with a once-in-a-blue-moon blip like that -- Hetzner saves me literally thousands of dollars per month in compute and bandwidth, I like doing [a limited amount of] sysadmin, and it's worth the trade-off. But, how do you think your CTO is going to feel about this move when all of your dev VMs go down just before a big customer deployment?
Another thing to consider, as a developer, is how to make your app as independent from vendor requirements. For example, if you build an app in such a way that it only works using SQS you’ll run into issues down the road.
Also, why y’all using dedicated dev VMs? IMHO it sounds like there are other issues to solve before you migrate the app somewhere else. If I was in your position I would work on making the app easier to setup and deploy. Your CTO has a lot to consider and being a CTO at a startup isn’t an easy thing. How long does it take to deliver a feature to customers? How can you help speed that up? What are the engineering goals of the CTO and how can you help with those goals? Make his or her life easier and work with them. Continuing to push an issue like this will cause you frustration and make your CTO dismissive of your ideas and suggestions in the future.
How many companies actually hit this stage? I can only think of a few, and usually it's because they have very specific hardware requirements (e.g. Dropbox's whole business is file storage, or if you're doing something that requires tons of GPUs).
Also, it’s about priorities and goals of the company. Security and control is the main reasons I see companies migrate to data centers. Generally things like GitHub Enterprise are being used.
In practical terms you should expect to never hit it. The point where such scaling (also the main value add of aws/azure) really matters and you start looking at an entire DC to lease, you've arrived in the realm of speculative fiction. You should not plan to get there, just as you should not plan to get a winning lottery ticket.