> PaaS is always going to offer less for the same price, but what surprised me was how start the difference can be. On Render.com, a 4GB RAM and 2vCPU service costs $85/mo. The same spec costs $14/mo on Scaleway (ARM).
I notice a trend that the people who scoff at hardware specs are usually the same ones standing in line for 2 cores and 4gb of RAM for $50+/month. They'll laugh when you suggest utilizing an obsolete $50 computer with a decade old CPU (that's just sitting in a closet), but more than willing to spend $50/month on similar performing hardware from a Cloud vendor.
I think people know they’re paying a premium on hardware.
The point of PaaS isn’t to get the best deal on specs it’s for small teams to iterate quickly and focus on product. If they’re successful they’ll outgrow it and hire people to manage infra.
I'm not gonna downvote you. But to your point about the hardware market, a mid range server CPU of today will be about as fast as an entry model CPU next year. Actual performance wise, your old server can probably keep up with a newer but slightly lower end server just fine. Obsolete !== useless.
I'm the kind of person who would rather take the 3y server and recomission it as a lower priority service than just do a 1:1 replacement. "Old" computers aren't as useless as they used to be. Computing power has advanced to the point where computation is arbitrary. For all intents, in most sectors, you can scale your compute capacity as high as your budget allows and you won't hit any performance barriers ever. There will always be more compute. It is a commodity now.
My stance is this; sure the new server is faster than the old one, but you know what's even faster? Dividing the existing workload between both of them.
> sure the new server is faster than the old one, but you know what's even faster? Dividing the existing workload between both of them.
How do you factor reliability into your analysis? I mean, the reason companies ditch their 3yo server is that their service life is nearing it's end, and odds are they will fail in the upcoming year or so.
Meanwhile, increased compute capacity means companies can use a single brand new server to replace multiple old ones, not to mention cheaper operational costs.
My 2024 goal was creating a website for me and hosting a copy of my newsletter there. I heard a lot of good things about Hetzner and choose them. Just right after signing up, even after I put a credit card with 3DS they immediately banned me. I wrote to support, they forwarded me to page. I uploaded my passport with my face. But my account is still banned. And worst thing is I’m 1 month late for my 2024 goal.
Try racknerd… they have various Black Friday/new year specials that rarely/never expire and the price can be as low as $25/year for a decent 2-cpu VPS. Not the same hardware as OVH, storage often limited, but still an interesting price point and service. Biggest limitations: no 2FA on the control panel and mounting an ISO requires talking to customer service… oh and no automated backups but they might have recently added snapshots.
I should add - keep your own backups. I can’t trust that a service this cheap will not lose my data somehow. It hasn’t yet, but with margins this thin I wonder if they’ll ever go out of business on me.
Hetzner can and will terminate your account and delete all your data on a whim, never telling you why. There is nothing at all you can do to prove your usage was legitimate. I get that they must deal with a lot of abuse but I wouldn't use them for anything mission critical ever.
> We’ve mounted the AX102 with 128 GB of DDR5 ECC RAM. In addition to the classic ECC (error correction code), which protects data both on the memory module and during transfer, the new DDR5 memory generation uses the on-die ECC method. This method carries out independent error correction directly on the DRAM chip, giving you greatly optimized reliability and data integrity.
Isn't on-die ECC necessary just because DDR5 is less reliable than previous generations, especially at higher frequencies and density? Makes me wonder whether they use an actual ECC DDR5 memory.
With comparable reliability, resilience, durability, and latency? I doubt scalability up or out is reasonable to ask, correct me if I’m underestimating its potential.
I have 14core/128GB RAM, 2x900GB nvme + 60TB spinning rust with 33TB traffic in someone else's closet and paying only $60 ¯\_(ツ)_/¯. Prices vary a lot when it comes to colocation
yeap - 60 for collocation only. 99.9% uptime SLA (power/network). I've had 5-minute blips 3 times in last 12 months. The hardware itself is bought used $400 for the whole server (not including HDD). HDD was around $400 too. so last 2 years: (24 * 60 + 800)/24 = ~$94/month and going down every month
Yup, I did this (not for $60!!) almost a decade ago with 4 savemyserver HP 1U’s and a switch. Each server cost me about $200 without HDD. Another $200 in ECC memory. I used it for a long time until colocation costs went up because they were bought by L3Harris.
I run a couple ryzen mini PCs on Verizon fios 1Gbps at home now for my hosting needs but I would jump at $60 colo in a heartbeat if I got even 100mb connection unmetered.
I am renting 16 core AMD with 128GB RAM and 2x2TB SSD for less then $100/month in Canadian pesos on Hetzner.
I also pay for 1Gbps symmetric fiber with dedicated IP going into my house ($140/month at the moment) so I also host some stuff right from my own place.
This is a bit exaggerated. I've had a 1U or 2U server box colo'ed for the last 20+ years, and it's been consistently in the $80-$100 range. Double my total costs if you add that I'm replacing a $5K server box every 5-7 years or so.
The duct-tape-maintenance vs your-time criticisms are 100% dead-on though.
Not just that, it's also all the legwork of managed hosting: all that OS configuration, patching, redundancy, testing, monitoring etc etc is someone else's job, and they are accountable. Plus other managed services like LB, DB, auth, etc etc. you might not want to duct tape yourself and manage every day. That cloud bill could be cheaper than your time is worth.
Plus flexibility of scale up / scale down as needed for load, transient testing, etc.
Cloud is definitely not for everyone but it makes sense for some.
> that OS configuration, patching, redundancy, testing, monitoring etc.
For larger projects I guess, but I have a few VPSes that have been running sites for a decade or more that require almost no maintenance (occasional apt-get upgrade). In fact it’s the frameworks / languages these projects were built in that cause most of the work (for which deployment target makes no difference).
With IaC it’s become feasible. You kinda iteratively build it up. With something more predictable like Nix it really gets close to shops run by a single person. And on e you start to get managed infrastructure, next question that pops up would often be which services do you want to delegate, and some folks wind up in serverless land—locked in after a while, which might be totally fine.
I've seen real-world connectivity, latency, and bandwidth problems crop up with enough frequency to be a real problem on a major budget "cloud" provider. It looked liked they'd badly cheaped out on their peering agreements.
Move the exact same workload to the industry's default but much more expensive choice, and the problems vanish entirely.
This is, unfortunately, one of those things that's really hard to judge about a hosting provider unless you have direct experience using them "at scale", as they say. Nobody puts that stuff on a sales page spec sheet or comparison grid.
Could I save money by hosting on real hardware at some popular, cheapish server-leasing place? Or colocating at one? Or hosting out of my own basement!? Maybe. Would it cause some users to consistently see dial-up speeds and dropped connections on gigabit Internet service because of either some quirk of routing, or bad peering agreements? Who knows!
I’ve observed those issues on the industry default CSPs. Terrible latencies for some services. Some of those budget CSPs simply resell AWS and Azure and GCP and OCP, while others run on commodity VPS infrastructure. And honestly there’s still too much ops work running workloads not on-premise, wasted time even though totally entertaining sometimes.
But isn't it more than just the vendor's hardware that one is paying for. Namely, the vendor's internet connection. Apologies if I am missing something obvious.
And for network redundancy / UPS / datacenter security / etc. Comparing a PaaS to some old PC running in a closet is completely missing the point.
If the thought of your service going down for a few hours because some random unplugged the power cable or your ADSL router crashed doesn’t make the CFO lose sleep then some raspberry pi is surely good enough. Just make sure you run it inside a safe if you store personal information.
They probably gained a lot of knowledge along the way, and if that knowledge happens to come in handy in your career, can you as easily put a price tag on that? I run services at home and spin up infra on AWS, but I'd say I learn the most from what I have at home.
That said, I barely need to maintain most of my home infrastructure. I have CI/CD scripts do the bulk of maintainership for me these days.
That’s knowledge forgotten in a week, unless you’re an ops side professional. You kinda set it up once and hope to forget to focus on your application layer.
It's much, much easier to recall knowledge that you've gained once, than knowledge that you gained 0 times. It has also not been my experience that I "hope to forget" that knowledge either. Perhaps you go about learning a different way than I do, though.
After all, can you really write a good application without knowledge of how it will be deployed, and the challenges users face deploying things on specific platforms (eg. Windows, baremetal linux, kubernetes, docker, etc)? I would argue that you'd often write naive applications that gimp itself in unexpected ways depending on how the user intends to use it, without that knowledge. Depending on what types of applications you tend to write, this might be less of a valuable point to you. For example a static site web dev probably wouldn't be as interested in the infrastructure, they just need a server that can bind on ports 80/443. But I see a lot of incredibly naive applications written by potentially naive software developers out there.
Totally agreed on your first sentence. Yes, I don’t want to keep in memory the minute details of how I set up the infrastructure or the platform. That’s why I have that in IaC format and have a dedicated operator on my team so they can deal with that on a daily basis I, while I do my work. I’m not “learning” ops, I try to delegate as much of it as I can, I have no choice. On my private projects I’ve always had to deal a lot with infrastructure, it’s experience accumulated over many years, and I feel quite unhappy about that waste of time. At work I learned that at some things I’m far more experienced and knowledgeable than our top ops people, while at others I had no idea. On such projects you must collaborate, but at the same time somehow make progress on your business logic. And let’s hope you’re not also responsible for the product.
Define “knowledge” you’re referring to. I have checklists for my ops. Define “really” and “good”. That sounds like perfectionist maxims without ground. Your listing of deployment targets can be extended by any BSD system, Android, iOS, IoT SoCs—what’s the point? If that’s your business requirement then you do it, if not then why bother?
If you think you can do both and create a “really good” app, or even product, well then you’re likely a genius. It depends on the depth, breadth and consistency of that “knowledge”—some people simply overestimate their capacity while others underestimate their potential. By poking around in Kubernetes is it deep enough for production or only broad enough to impress someone who’s deeper yet not broader? By reading the docs and textbooks and learning from people, would that be enough for production?
I said I hoped to forget that ops stuff because I have other duties to attend to and there’s too much variance and unreliability in those ops tasks, you need to make a context switch, hence IaC, hence managed, so you as a non-operator can focus on your tasks. Infrastructure and platforms are commodity goods now.
> By poking around in Kubernetes is it deep enough for production or only broad enough to impress someone who’s deeper yet not broader? By reading the docs and textbooks and learning from people, would that be enough for production?
Many enterprises and companies use Kubernetes as at least _one_ of their deployment targets. So if you're a provider of software that has paying companies consuming it, does it matter if kubernetes is "deep enough for production"? It's being used in production, so I would argue, yes, from the perspective of a software provider, kubernetes is in fact deep enough for production. Though to one of your other points, define deep enough, because that statement seems like a moving target depending on who you talk to. But in general, if a piece of software doesn't work well in kubernetes, it's because the authors of that software hadn't considered any setup other than running as a monolith systemd service with an sqlite db for state. That will move me into the next quote:
> Define “knowledge” you’re referring to. I have checklists for my ops. Define “really” and “good”. That sounds like perfectionist maxims without ground. Your listing of deployment targets can be extended by any BSD system, Android, iOS, IoT SoCs—what’s the point? If that’s your business requirement then you do it, if not then why bother?
Knowledge is a fairly generic term, isn't it. It's a placeholder word for a series (or bucket) of facts about one or several domains. "Really" is a nuanced word that means different things depending on how it's used. For example, in my sentence fragment, "can you really write," I'm using the word 'really' to express disbelief in one's ability to perform the context. On the word "good" we can use context clues to extrapolate that I probably am using the word "good" to mean one that supports a wide variety of deployment targets with ease because you often do not know how your product consumer's infrastructure looks, but you can probably guess that if its intended deployment target is a linux server, these days, it should probably also be well supported in Docker and Kubernetes. So that is "good." There are a bucket of other concepts that make a piece of software "good" as well that I'll merely touch on for the sake of brevity, such as being well tested, generally lacking unchecked exceptions, runtime panics, is well performant when ran for long periods of time, etc.
Though, to be honest, I think you probably could have used context clues to come up with rough definitions of those words yourself, and I suppose your purpose was to glean what I think those words mean. In my opinion, the exercise seemed a bit more tedious than maintaining infrastructure though.
I think this is a pretty fair take in theory. In practice, I feel that I work with a lot of people that excel in one very specific domain of knowledge and lack anything else, probably because they use this exact mindset. And when they uncover a situation they have no awareness of, they rely on me to fix it, or spend 3 additional days studying the problem while it is a problem. And the thing about problems is, they're kind of nice to be able to solve quickly. Though it would be fair to retort that I'm probably a jack of all (or many) trades, but a master of none, which sometimes hurts my ability to get work done. Though, in my honest opinion, I feel I come out on top in most situations.
Anyway, this is a pick your poison scenario, probably. So, don't let me try to tell you one way is the best way.
My preferred way of looking at this is “your project costs you this much to keep being alive”. An upfront cost means maintenance costs are essentially zero, resulting in your projects needing to hit a much lower bar to stay alive.
It's zen-like "motorcycle maintenance" for me, and keeps me abreast of what other Ops folks deal with in this space.
It has probably contributed zero directly to my software engineering career, however, there are moments where a deep understanding of the quicksand under my app foundations can help, and shortcut strange debugging sessions and the like.
The time I spend is recreational for me. I can see where others, particularly Ops professionals, are horrified at the idea of doing lib and OS maintenance/updating for fun. It's a very yuckable yum :D
I've recently gone the other way. I was self-hosting everything on a DigitalOcean VPS, but keeping the OS maintained, and indeed the headaches of configuring Nginx, letsencrypt, postgres and so on became more annoying not less each time I wanted to make a new app, because every app was a bit different.
I'm now running my primary project on Fly.io and I'm pretty happy with it overall.
"No matter how small is the service, no matter how rarely you need it, you’d need to pay the full price."
On Fly.io I'm running an app server, a db server, and another app server with a 1GB volume for a Discord bot. Everything fits in the free plan.
The thing about PaaS is you really have to do your research. It's not like VPS providers where all you really need to look at is how much compute and storage you get for a monthly price. PaaS have a lot more subtleties and yes, it happens that the startups behind them sometimes blow up or get bought out by huge public enterprise companies. VPS providers tend to be lower risk.
The tradeoff is worth it for me, but it really depends on your skillset, your priorities and so on. I can maintain a VPS, but I have very limited time, so I want to focus every spare hour I have on developing my product.
I've been amazed at the resilience and convenience of a handful of shell scripts calling "docker" commands on Debian for my server at home.
- Figured I'd need to screw with Systemd at some point. Nope, whatever Docker's doing restarts my services on a system restart, and auto-restarts if they break. I haven't had to lift a finger for any of that. My services are always just there, unless something really goes horribly wrong.
- Which directories I need to backup is documented in the shell scripts themselves. Very clear and easy.
- Moving those directories and my shell scripts to another server, potentially with a different distro, would be trivial. Rsync two directories (I've put all the directories I mount in the docker images, under a single directory for convenience), shell in, run the scripts. Writing a meta-script to run all of them would be easy. On a VPS I could have everything that mattered on a network drive, and that'd make it even simpler. Mount network drive, run script(s).
- Version updates are easy. I can switch between "use the latest" and "use this specific version until I say otherwise" at will. Rollbacks are trivial. If the services were public-facing I could automate a lot of this with maybe an hour of effort.
- Port mapping's covered by Docker. If these were public-facing it'd be pretty easy to add one extra container for SSL certs and termination (probably Caddy, because I'm lazy, though historically my answer for this at paying gigs has been haproxy). Like, truly, the degree to which I can interact with and configure this system entirely by using portable-everywhere docker commands & config is very high.
I've been running servers (sometimes private, sometimes public) at home since like 2000, and this is easily my favorite approach I've used so far.
I've used stuff like Dokku at work. I dunno—it's another thing that can and does break. If you're just self-hosting a few services and aren't trying to coordinate the work of several developers, IMO it's simpler and not-slower to just use Docker directly.
I just mean in the usual way that another component in the stack is another way for things to go wrong, another thing to update that might mean api changes that require further effort, and so on.
It was plenty solid, but I’ve definitely seen it fall victim to operator error :-) I’d certainly consider using it again to support a team, under the right circumstances.
Interestingly, I have a very different experience than you. I simply have a script that sets up the server. I keep updating the script to make it better each time I hit an edge case. At this point, it's pretty bullet-proof. Updates are automatic and so I can leave the server running for months without any intervention.
Due to my love to shiny things, I keep wanting to find an excuse to move to a PaaS but I can never find a sufficient justification.
Honestly, I can't remember exactly. I believe there are some settings in Ubuntu that apply updates automatically. I set just the security updates to reduce the risk of breaking something. The beauty of the system is that I can forget stuff like this and still get the benefit of my previous knowledge.
It's basically a combination of documentation (Step 1: Do this, Step 2 Do that) and shell scripts wherever possible. I can just follow the steps and run the commands without thinking and go from a bare server to a fully deployed working app in about 20 minutes.
> The thing about PaaS is you really have to do your research.
I find this true of pretty much all modern cloud development. So many people want to just pretend it’s another VPS alternative and are shocked that their misuse leads to high monthly bills. You need different patterns to take advantage of the strengths of PaaS cloud services and to avoid the weaknesses and gotchas. I PaaS all of the things that I can and my maintenance and support efforts have never been lower. My services scale to zero and I only pay for actual utilization.
If you’re standing up a lot of VMs in the cloud and pretending it’s just another data center of course you’re going to have a bad time and waste money.
What's the common definition of self-hosting these days?
I've always considered self-hosting to mean I'm managing hardware, but its clear the author here sees it more as a self-managed OS and infrastructure.
It actually feels very similar to the whole MPA vs SPA debate in web development. Maybe I'm just getting old, but self-hosting and SPA have specific meanings I learned years ago that seem to be getting redefined now rather than coming up with new names.
On-prem is a really specific term for onsite hardware. Though I don't think its as clearly accepted that renting a VPS is self-hosting.
If that is the meaning now that is a more modern meaning, in my experience VPS was not part of a self-hosting concept before cloud providers were so common. At that time the options were your own hardware or a rented VPS, they couldn't both be self-hosting. Today that's less clear, hence my question of what is the common meaning today.
> The practice of self-hosting web services became more feasible with the development of cloud computing and virtualization technologies, which enabled users to run their own servers on remote hardware or virtual machines.
If you buy their definition, it seems inline with the idea that self hosting is more about the software than the hardware these days.
Self-hosting doesn’t necessarily imply that it’s on-premises, or that you own the hardware. It means that you are fully managing the software side of things (everything that is “on the host”) and have full control over that.
This thread is the first time I'm hearing this definition. Self hosting has always meant using your own hardware. Cloud/VPS/VMs/shared hosting or whatever else have never qualified.
For me it’s the opposite: VPS are the standard solution, since it’s difficult to self-host behind a consumer ISP. Wikipedia seems to agree [0]: “The practice of self-hosting web services became more feasible with the development of cloud computing and virtualization technologies, which enabled users to run their own servers on remote hardware or virtual machines.”
The original difference was between using a service like WordPress vs. running an instance of the WordPress software yourself. Who owns the hardware it’s running on or where it is located is largely irrelevant for the definition.
I always viewed self-hosting as "shades of gray." The person running Wordpress on a VPS or EC2 instance, as opposed to Wordpress.com, is self-hosting. They have to configure the software. The person running it on a bare metal server in his basement, even more so: both software and hardware. They still both qualify, one is just more extreme.
This presupposes that you need autoscaling. I suspect the vast majority of applications have pretty predictable resource requirements / growth characteristics. Over-provisioning your hardware a bit to handle anticipated spikes is probably fine for 80%+ of use-cases.
Good on the author, but using a Virtual Private Server skirts very close to not self-hosting. When I read 'self-hosting', I imagined buying/building a physical server and either putting it into a datacenter or running it in your home.
Lately I've been thinking of creating a bare-bones HTML website of my own and maybe I'll run it on a Raspberry Pi at home. I think that would qualify as 'self-hosting'.
Between bots and the Reddit/HN hug of death, if you ever do this don't ever advertise your website (to avoid getting DoS'd) and put some firewall in front.
Moving from a PaaS to a VPS is 95% the same amount of effort and energy as spinning you own rust under your desk. Semantics matter but holding a definition to “need to suffer a power supply failure to count” isnt really necessary.
It makes little difference who owns the hardware in the data center. The important thing is who selects, controls, and manages the software that runs on it. Therefore that’s the main dividing line.
I wonder what people host that require such specs.
I host on a skylake about 7 years old I think, with 12 GB RAM. About 30 docker images running Home Assistant, Bitwarden, the *arr suite, jellyfin, minecraft ... Nothing fancy and I have heaps of free CPU and RAM.
I understand that one can easily load CPUs with compute processing, or RAM with video transformation but for a generic self-hoster of typical services I am surprised by the typical setup people have (which is great for them, I am just curious)
I'm surprised no one has brought up the power cost point yet. I have a couple services I'd like to be up 24/7 and paying for hosting on them is actually cheaper than I'd be able to do from home just due to the cost of electricity where I live. Plus I've had quite a few ISP outages, but my provider, as far as I can tell, has been up 24/7/365 in the few years I've been paying them.
Docker and Kubernetes really are the two best things happened to me in my selfhosting journey. They made billion dollar enterprise grade tech approachable in my homelab.
- I powered on a brand new mini PC, 10 minutes later it showed up as a node in my cluster and started running some containers
- Two servers died but I didn't notice until a month later, because everything just kept working.
- Some database file got corrupted but the cluster automatically fixed it from a replica on another node.
- I almost completely forget how to manage certs with letsencrypt, because the system will never miss the renewal window unless the very last server in my lab goes down.
Add VPN to that list. It used to be a monumental pain to set up a home network, but now with something like Wireguard/Tailscale/ZeroTier/Nebula you can do it in a few clicks.
Yes, but once you know how to use it, everything else is an annoying toy. Also, of course, it being the standard means there is no end of tooling designed to support it.
I can’t count the number of times I’ve been frustrated at my current job because I have to wade through layers of ECS bullshit to do something.
All that said, if you don’t need containers, turns out you can get a lot done with two servers behind HAProxy + keepalived, running stuff with systemd.
I've been pretty sour on Kubernetes, but this looks like it could bring some of the niceties of PaaS to k8s. I haven't looked into the deployment process yet though, perhaps that's where all the pain lies
You may enjoy learning and using different implementations for networking, storage, DNS, containerization...from time to time.
But I just learnt Kubernetes and suddenly all those implementations become interchangeable and my knowledge is transferable between homelab and work, and even across companies and platforms.
Of course, because very few projects need every feature of Kubernetes. But most projects need three or four of them, and many projects need something from the long tail too.
I’ve been running my personal cluster happily on k8s for years.
These are great operational wins. Agreed very much that having autonomic (can fix itself) systems at your back is a massive game changer. De-crustifies the act of running things.
The other win is that there's a substantial cultural base to this way to go. Folks have been doing selfhosting for ages, but everyone has their own boutique setup some their way. A couple tools and techniques could be shared, but mostly everyone took blank slate configs & built their own system up, & added their own monitoring & operational scripts.
https://github.com/onedr0p/home-ops is a set of helm scripts and other tools that is widely widely used, and there's a lot more like it. It's a huge build out, using convention and a common platform to enable portable knowledge & sharing.
Self hosting did not have intellectual scale out at it's back, before Kubernetes came along. Docker and ansible and others have been around, but theres never been remotely the success there has been today in empowering users to setup & run complex services.
We really have clawed out of the server-hugging jungle &started building some villages. It's wonderful to see.
IMHO they are the same wins. Because behind all these, the real value is standardization. Kubernetes offers standard APIs and all the vendors fall in line with their own implementations.
As a result, the software I run in my homelab for free, is the same software battle-tested in various enterprise environments, from 5-person startups to planet-scale mega corps. There are paid engineers and companies out there making serious long term commitment to it. This is truly amazing.
I really like your first post, because it highlights so many real fantastic wins, about how life is different & how the system really helps.
And I felt like we have had so many promises from various bits of software & professional teams about how they will heal and help us. But it hasn't planned out. The Mesos of the world, the OpenStacks of the world: they all have made similar claims to what you are saying, but something is so different this time. It's working. It's adding up.
Folks absolutely try to throw the "complexity" of it under the bus, but there's something incredibly simple about the pattern here of using apiserver to state your desires, and letting various controllers/operators work it all out, keep it going. It's a simple, repeatable pattern.
And the consistency of it all allows for systems like Helm to emerge. Not even great, but ok, adequate and demonstrating how portable the consistent platform of Kubernetes really is; it just works.
Something is different this time, even though so much sounds the same as previous dances around. It sounds complex, but many many people can get more onboard, the scope is more right sized to fit all the concerns, and the way that asking for things is just making a manifest is so much more accessible than before.
What is different this time? Why is this working this time? Why are there such more vibrant communities?
I would call this divide more-so serverless vs servers than self hosting. I"ve recently gone the opposite direction. I used to think serverless was too exepensive and unpredictable but after almost a decade of Go and 2 years of Rust I'm really coming the opinion that the resources required to run services are completely warped.
2 cores and 4GB ram is the recommended specs to run the 2008 game Crysis. It's hard to imagine that moving JSON around (which is probably 95% of services) is more demanding than Crysis. As I've begun really paying attention to the services being deployed you can actually get a lot of mileage out of the free tier especially now that they support running native binaries.
108 comments
[ 3.4 ms ] story [ 203 ms ] threadI notice a trend that the people who scoff at hardware specs are usually the same ones standing in line for 2 cores and 4gb of RAM for $50+/month. They'll laugh when you suggest utilizing an obsolete $50 computer with a decade old CPU (that's just sitting in a closet), but more than willing to spend $50/month on similar performing hardware from a Cloud vendor.
1. Those who love their Macs a little too much
2. Those who routinely build a PC.
i.e. Are you familiar with the hardware market.
I'm the kind of person who would rather take the 3y server and recomission it as a lower priority service than just do a 1:1 replacement. "Old" computers aren't as useless as they used to be. Computing power has advanced to the point where computation is arbitrary. For all intents, in most sectors, you can scale your compute capacity as high as your budget allows and you won't hit any performance barriers ever. There will always be more compute. It is a commodity now.
My stance is this; sure the new server is faster than the old one, but you know what's even faster? Dividing the existing workload between both of them.
What about the core count and memory capacity
How do you factor reliability into your analysis? I mean, the reason companies ditch their 3yo server is that their service life is nearing it's end, and odds are they will fail in the upcoming year or so.
Meanwhile, increased compute capacity means companies can use a single brand new server to replace multiple old ones, not to mention cheaper operational costs.
Unbelievable (new) specs, hosted and at prices points tough to beat doing it yourself.
E.g.
Hetzner
For only ~$100/mohttps://www.hetzner.com/news/new-amd-ryzen-7950-server
They normally are trying to keep their IPs and Network clean of scammers abusing their resources, which inevitably hurt all customers.
Is there still more needed to complete their KYC process?
AWS is expensive and not without trouble but in my experience they’ve been best of the pack.
Otherwise, on-premises + collocation is never a mistake if the maintenance and upright expenditure is in your budget.
Isn't on-die ECC necessary just because DDR5 is less reliable than previous generations, especially at higher frequencies and density? Makes me wonder whether they use an actual ECC DDR5 memory.
Yet many other folks praise them…
What’s your experience?
I run a couple ryzen mini PCs on Verizon fios 1Gbps at home now for my hosting needs but I would jump at $60 colo in a heartbeat if I got even 100mb connection unmetered.
I also pay for 1Gbps symmetric fiber with dedicated IP going into my house ($140/month at the moment) so I also host some stuff right from my own place.
The duct-tape-maintenance vs your-time criticisms are 100% dead-on though.
Plus flexibility of scale up / scale down as needed for load, transient testing, etc.
Cloud is definitely not for everyone but it makes sense for some.
For larger projects I guess, but I have a few VPSes that have been running sites for a decade or more that require almost no maintenance (occasional apt-get upgrade). In fact it’s the frameworks / languages these projects were built in that cause most of the work (for which deployment target makes no difference).
Move the exact same workload to the industry's default but much more expensive choice, and the problems vanish entirely.
This is, unfortunately, one of those things that's really hard to judge about a hosting provider unless you have direct experience using them "at scale", as they say. Nobody puts that stuff on a sales page spec sheet or comparison grid.
Could I save money by hosting on real hardware at some popular, cheapish server-leasing place? Or colocating at one? Or hosting out of my own basement!? Maybe. Would it cause some users to consistently see dial-up speeds and dropped connections on gigabit Internet service because of either some quirk of routing, or bad peering agreements? Who knows!
If the thought of your service going down for a few hours because some random unplugged the power cable or your ADSL router crashed doesn’t make the CFO lose sleep then some raspberry pi is surely good enough. Just make sure you run it inside a safe if you store personal information.
That said, I barely need to maintain most of my home infrastructure. I have CI/CD scripts do the bulk of maintainership for me these days.
After all, can you really write a good application without knowledge of how it will be deployed, and the challenges users face deploying things on specific platforms (eg. Windows, baremetal linux, kubernetes, docker, etc)? I would argue that you'd often write naive applications that gimp itself in unexpected ways depending on how the user intends to use it, without that knowledge. Depending on what types of applications you tend to write, this might be less of a valuable point to you. For example a static site web dev probably wouldn't be as interested in the infrastructure, they just need a server that can bind on ports 80/443. But I see a lot of incredibly naive applications written by potentially naive software developers out there.
Define “knowledge” you’re referring to. I have checklists for my ops. Define “really” and “good”. That sounds like perfectionist maxims without ground. Your listing of deployment targets can be extended by any BSD system, Android, iOS, IoT SoCs—what’s the point? If that’s your business requirement then you do it, if not then why bother?
If you think you can do both and create a “really good” app, or even product, well then you’re likely a genius. It depends on the depth, breadth and consistency of that “knowledge”—some people simply overestimate their capacity while others underestimate their potential. By poking around in Kubernetes is it deep enough for production or only broad enough to impress someone who’s deeper yet not broader? By reading the docs and textbooks and learning from people, would that be enough for production?
I said I hoped to forget that ops stuff because I have other duties to attend to and there’s too much variance and unreliability in those ops tasks, you need to make a context switch, hence IaC, hence managed, so you as a non-operator can focus on your tasks. Infrastructure and platforms are commodity goods now.
Many enterprises and companies use Kubernetes as at least _one_ of their deployment targets. So if you're a provider of software that has paying companies consuming it, does it matter if kubernetes is "deep enough for production"? It's being used in production, so I would argue, yes, from the perspective of a software provider, kubernetes is in fact deep enough for production. Though to one of your other points, define deep enough, because that statement seems like a moving target depending on who you talk to. But in general, if a piece of software doesn't work well in kubernetes, it's because the authors of that software hadn't considered any setup other than running as a monolith systemd service with an sqlite db for state. That will move me into the next quote:
> Define “knowledge” you’re referring to. I have checklists for my ops. Define “really” and “good”. That sounds like perfectionist maxims without ground. Your listing of deployment targets can be extended by any BSD system, Android, iOS, IoT SoCs—what’s the point? If that’s your business requirement then you do it, if not then why bother?
Knowledge is a fairly generic term, isn't it. It's a placeholder word for a series (or bucket) of facts about one or several domains. "Really" is a nuanced word that means different things depending on how it's used. For example, in my sentence fragment, "can you really write," I'm using the word 'really' to express disbelief in one's ability to perform the context. On the word "good" we can use context clues to extrapolate that I probably am using the word "good" to mean one that supports a wide variety of deployment targets with ease because you often do not know how your product consumer's infrastructure looks, but you can probably guess that if its intended deployment target is a linux server, these days, it should probably also be well supported in Docker and Kubernetes. So that is "good." There are a bucket of other concepts that make a piece of software "good" as well that I'll merely touch on for the sake of brevity, such as being well tested, generally lacking unchecked exceptions, runtime panics, is well performant when ran for long periods of time, etc.
Though, to be honest, I think you probably could have used context clues to come up with rough definitions of those words yourself, and I suppose your purpose was to glean what I think those words mean. In my opinion, the exercise seemed a bit more tedious than maintaining infrastructure though.
Well, yeah, I’d rather not waste my time. Time is money.
Anyway, this is a pick your poison scenario, probably. So, don't let me try to tell you one way is the best way.
It has probably contributed zero directly to my software engineering career, however, there are moments where a deep understanding of the quicksand under my app foundations can help, and shortcut strange debugging sessions and the like.
The time I spend is recreational for me. I can see where others, particularly Ops professionals, are horrified at the idea of doing lib and OS maintenance/updating for fun. It's a very yuckable yum :D
“Oh, I remember doing this a year ago in my homelab… it was a mistake.”
Otherwise, that time is his free time and he can do what he likes.
I get triggered by it because I get people in my company coming to me we should switch to cloud - but we are in cloud only that it is IaaS.
I'm now running my primary project on Fly.io and I'm pretty happy with it overall.
"No matter how small is the service, no matter how rarely you need it, you’d need to pay the full price."
On Fly.io I'm running an app server, a db server, and another app server with a 1GB volume for a Discord bot. Everything fits in the free plan.
The thing about PaaS is you really have to do your research. It's not like VPS providers where all you really need to look at is how much compute and storage you get for a monthly price. PaaS have a lot more subtleties and yes, it happens that the startups behind them sometimes blow up or get bought out by huge public enterprise companies. VPS providers tend to be lower risk.
The tradeoff is worth it for me, but it really depends on your skillset, your priorities and so on. I can maintain a VPS, but I have very limited time, so I want to focus every spare hour I have on developing my product.
- Figured I'd need to screw with Systemd at some point. Nope, whatever Docker's doing restarts my services on a system restart, and auto-restarts if they break. I haven't had to lift a finger for any of that. My services are always just there, unless something really goes horribly wrong.
- Which directories I need to backup is documented in the shell scripts themselves. Very clear and easy.
- Moving those directories and my shell scripts to another server, potentially with a different distro, would be trivial. Rsync two directories (I've put all the directories I mount in the docker images, under a single directory for convenience), shell in, run the scripts. Writing a meta-script to run all of them would be easy. On a VPS I could have everything that mattered on a network drive, and that'd make it even simpler. Mount network drive, run script(s).
- Version updates are easy. I can switch between "use the latest" and "use this specific version until I say otherwise" at will. Rollbacks are trivial. If the services were public-facing I could automate a lot of this with maybe an hour of effort.
- Port mapping's covered by Docker. If these were public-facing it'd be pretty easy to add one extra container for SSL certs and termination (probably Caddy, because I'm lazy, though historically my answer for this at paying gigs has been haproxy). Like, truly, the degree to which I can interact with and configure this system entirely by using portable-everywhere docker commands & config is very high.
I've been running servers (sometimes private, sometimes public) at home since like 2000, and this is easily my favorite approach I've used so far.
I've used stuff like Dokku at work. I dunno—it's another thing that can and does break. If you're just self-hosting a few services and aren't trying to coordinate the work of several developers, IMO it's simpler and not-slower to just use Docker directly.
Would love to hear more about how Dokku broke as it will help me polish the project further :)
It was plenty solid, but I’ve definitely seen it fall victim to operator error :-) I’d certainly consider using it again to support a team, under the right circumstances.
Due to my love to shiny things, I keep wanting to find an excuse to move to a PaaS but I can never find a sufficient justification.
I find this true of pretty much all modern cloud development. So many people want to just pretend it’s another VPS alternative and are shocked that their misuse leads to high monthly bills. You need different patterns to take advantage of the strengths of PaaS cloud services and to avoid the weaknesses and gotchas. I PaaS all of the things that I can and my maintenance and support efforts have never been lower. My services scale to zero and I only pay for actual utilization.
If you’re standing up a lot of VMs in the cloud and pretending it’s just another data center of course you’re going to have a bad time and waste money.
I've always considered self-hosting to mean I'm managing hardware, but its clear the author here sees it more as a self-managed OS and infrastructure.
It actually feels very similar to the whole MPA vs SPA debate in web development. Maybe I'm just getting old, but self-hosting and SPA have specific meanings I learned years ago that seem to be getting redefined now rather than coming up with new names.
If you wanted to communicate that you're dealing with hardware I would imagine you would say co-locating or talk about your datacenter.
If that is the meaning now that is a more modern meaning, in my experience VPS was not part of a self-hosting concept before cloud providers were so common. At that time the options were your own hardware or a rented VPS, they couldn't both be self-hosting. Today that's less clear, hence my question of what is the common meaning today.
https://en.wikipedia.org/wiki/Self-hosting_(web_services)
> The practice of self-hosting web services became more feasible with the development of cloud computing and virtualization technologies, which enabled users to run their own servers on remote hardware or virtual machines.
If you buy their definition, it seems inline with the idea that self hosting is more about the software than the hardware these days.
I was a little surprised by the article but maybe it is just a redefinition over time as the industry changes.
[0] https://en.wikipedia.org/wiki/Self-hosting_(web_services)
The original difference was between using a service like WordPress vs. running an instance of the WordPress software yourself. Who owns the hardware it’s running on or where it is located is largely irrelevant for the definition.
Here is another reference: https://www.computerhope.com/jargon/s/self-hosting.htm
Self hosting is your hardware in your house/property with your software.
but times change i guess. I do understand why that definition would change. but I feel we should maybe name it slightly differently.
Lately I've been thinking of creating a bare-bones HTML website of my own and maybe I'll run it on a Raspberry Pi at home. I think that would qualify as 'self-hosting'.
I host on a skylake about 7 years old I think, with 12 GB RAM. About 30 docker images running Home Assistant, Bitwarden, the *arr suite, jellyfin, minecraft ... Nothing fancy and I have heaps of free CPU and RAM.
I understand that one can easily load CPUs with compute processing, or RAM with video transformation but for a generic self-hoster of typical services I am surprised by the typical setup people have (which is great for them, I am just curious)
> Self-hosting has become more reliable
Docker and Kubernetes really are the two best things happened to me in my selfhosting journey. They made billion dollar enterprise grade tech approachable in my homelab.
- I powered on a brand new mini PC, 10 minutes later it showed up as a node in my cluster and started running some containers
- Two servers died but I didn't notice until a month later, because everything just kept working.
- Some database file got corrupted but the cluster automatically fixed it from a replica on another node.
- I almost completely forget how to manage certs with letsencrypt, because the system will never miss the renewal window unless the very last server in my lab goes down.
I can’t count the number of times I’ve been frustrated at my current job because I have to wade through layers of ECS bullshit to do something.
All that said, if you don’t need containers, turns out you can get a lot done with two servers behind HAProxy + keepalived, running stuff with systemd.
I've been pretty sour on Kubernetes, but this looks like it could bring some of the niceties of PaaS to k8s. I haven't looked into the deployment process yet though, perhaps that's where all the pain lies
But I just learnt Kubernetes and suddenly all those implementations become interchangeable and my knowledge is transferable between homelab and work, and even across companies and platforms.
I’ve been running my personal cluster happily on k8s for years.
The other win is that there's a substantial cultural base to this way to go. Folks have been doing selfhosting for ages, but everyone has their own boutique setup some their way. A couple tools and techniques could be shared, but mostly everyone took blank slate configs & built their own system up, & added their own monitoring & operational scripts.
https://github.com/onedr0p/home-ops is a set of helm scripts and other tools that is widely widely used, and there's a lot more like it. It's a huge build out, using convention and a common platform to enable portable knowledge & sharing.
Self hosting did not have intellectual scale out at it's back, before Kubernetes came along. Docker and ansible and others have been around, but theres never been remotely the success there has been today in empowering users to setup & run complex services.
We really have clawed out of the server-hugging jungle &started building some villages. It's wonderful to see.
As a result, the software I run in my homelab for free, is the same software battle-tested in various enterprise environments, from 5-person startups to planet-scale mega corps. There are paid engineers and companies out there making serious long term commitment to it. This is truly amazing.
And I felt like we have had so many promises from various bits of software & professional teams about how they will heal and help us. But it hasn't planned out. The Mesos of the world, the OpenStacks of the world: they all have made similar claims to what you are saying, but something is so different this time. It's working. It's adding up.
Folks absolutely try to throw the "complexity" of it under the bus, but there's something incredibly simple about the pattern here of using apiserver to state your desires, and letting various controllers/operators work it all out, keep it going. It's a simple, repeatable pattern.
And the consistency of it all allows for systems like Helm to emerge. Not even great, but ok, adequate and demonstrating how portable the consistent platform of Kubernetes really is; it just works.
Something is different this time, even though so much sounds the same as previous dances around. It sounds complex, but many many people can get more onboard, the scope is more right sized to fit all the concerns, and the way that asking for things is just making a manifest is so much more accessible than before.
What is different this time? Why is this working this time? Why are there such more vibrant communities?
2 cores and 4GB ram is the recommended specs to run the 2008 game Crysis. It's hard to imagine that moving JSON around (which is probably 95% of services) is more demanding than Crysis. As I've begun really paying attention to the services being deployed you can actually get a lot of mileage out of the free tier especially now that they support running native binaries.