HN Explorer

My SaaS Journey from $1000/Mo Heroku to Home Server

22 points by campervans ↗ HN
I have a production SaaS product, used by 1,000's a day.

Server bills are $1,000/month. Heroku mostly and Digital Ocean.

I'm thinking of moving to a home server, setting up 2 servers (a Staging + Production), with 4 raspberry PI 5's as worker servers.

It seems I can get far better performance, with a < 3 Month payback period...

What am I missing??

Server spec:

Server ->https://pcbuilder.net/rigs/4sg24H/ - $988 /each x 2 + Raspberry PI 5 ->https://www.raspberrypi.com/products/raspberry-pi-5/#:~:text=Buy%20now-,Specification,-Broadcom%20BCM2712%202.4GHz - $80 /each x 4 + Hardware Firewall -> https://www.amazon.com/Fortinet-FortiGate-Firewall-Throughput-Protection/dp/B07ZZMFWJ7/ - $350 x 1 + Switch -> https://www.amazon.com/NETGEAR-5-Port-Gigabit-Ethernet-Managed/dp/B07PJ7XZ7X - $25 x 1

Total = $2,671 (+ time and effort to config) + DataDog

Payback < 3 months

Power cuts - Not had one ever. But, could spin up a server if a ping of the home server goes down Internet disruption - Not ever happened. But would have a 5G fall back router.

What am I missing?? Why is this not more common?

32 comments

[ 3.1 ms ] story [ 77.4 ms ] thread
TacticalCoder
> Power cuts - Not had one ever.

I had both blackouts and brownouts. They're typically very short lived so why not add a little UPS to the mix?

Regarding the Pis: I've got both an army of Pi and now an army of NUCs. I don't know what'd work best for you.

Also is Gbit/s sufficient?

> Payback < 3 months

To me it's a no brainer but you could also aim for a payback in less than, say, six months and have something really smoking that $1000/month cloud offering.

campervans
NUC's look interesting, thanks for this!

Yeh, bandwidth should be more than sufficient.

Agreed, looking at bleeding edge SDD's, seems a simple way to have a blazing fast DB. Even a 12 month payback works. We're feature complete so I could keep the setup without upgrades for some time

bcrl
Trusting bleeding edge consumer SSDs would be a bad idea in my opinion. Go read some of the threads over on realworldtech.com of folks reporting failure to detect bit errors in consumer SSDs. I'd have similar concerns with using Raspberry Pis that do not have ECC memory. These things don't matter until they do. Once you've wasted a day or 2 looking for a kernel bug that turned out to be a hardware issue, the value of error detection in your hardware becomes well worth it... Unless you don't value your own time.
ThePhysicist
Compliance, data protection? Not sure if you just don't care about these or if your service doesn't process any sensitive or personal data but having all your customer data sitting at home in your basement or living room where it can be easily stolen sounds like a nightmare, and will put you in a bad spot in case something happens. Not sure if that's relevant to you and where you're based, I would never risk that though, and I think it's quite unprofessional.
malteg
can you share some more information about the current setup and pricing.

I would definitly not go for a complete home setup - but a hybrid approach is quite feasable - eg. worker nodes, staging env and a vault hosting inhouse. but for production for me personally it would be too risky. you should also calculate your own hourly rate what it cost to maintain this setup...

campervans
I have 2x Web Dynos (Standard) and 2x Web Dynos (Performance). 2 x 2x standard Worker Dynos. Then 2 x DO 16GB 4 core CPU's.

Then have about $100 in excess cost on addons on Heroku (vs configing them directly)

malteg
Did you had a look at contabo?
campervans
This is a great option too.

I went for Hetzner in the end + Coolify

throwaway828
Dedicated server?

Take a look at serverhunter.com

tomcam
Hey, thanks. Didn’t know it existed
dsr_
Most residential ISPs forbid commercial use.

Many residential ISPs won't give you a static IPv4 address.

Most residential ISPs won't give you sufficiently symmetrical bandwidth.

VM hosting companies have spare hardware on hand, and can generally shift you either transparently or with minimal downtime to different hardware.

If none of these things are an issue for you, go ahead.

Personally, I wouldn't use RPi hardware, EVO 970s, 2TB spinners, or a Fortigate.

Four boxes, each with a 12 core CPU, 64 GB RAM, a 2TB NVMe with at least a 1200TBW lifetime, and a pair of 4-8 TB spinners in RAID1. One box runs staging including workers, one runs production including workers, one runs all your firewall, utility, administration and backups, and the fourth is configured and powered off -- once a month, power it up, bring it up to date, and shut it down. Someday you will need it as a spare or for capacity. Get extra NICs for all of them.

I would estimate $5K in hardware costs. Buy an online UPS and connect the production and utility boxes to it.

GianFabien
I agree with most of your points. Without knowing more about the current workloads and projected growth, it seems that the servers you suggest could be far more powerful than necessary. Of course, if you can afford to, then over-provisioning is a prudent measure.
campervans
Thanks, I think over provisioning should reduce the chance of part failure.

Was also considering underclocking the CPU, but not sure if it would be helpful.

campervans
Ace, thanks for this!

This is a great setup, and still sub 6 month payback. For comparison I have 2.5GB or less on my Heroku servers!

IP address management is definitely food for thought.

I might to a dry run with a PC and test, before investing in the full setup

Jtsummers
If you're going to have the 4th box as a backup, instead of power cycling and updating it, consider exercising that backup. Each <period of time> move one system (since you've described it with identical configurations across all four) to the backup, shutdown (or put into some other idle mode) the now unused system. Repeat. This exercises your failover/restoration process. If you never do that, you don't have a failover/backup process.
GianFabien
Is your home internet connection fast, reliable and provide sufficient volumes? Many connections are not far slower for outgres than ingres. Does your ISP permit the usage you propose under your current contract? What about IP, can you get static IP, is it within the ISP's range, DNS? I doubt that dynamic DNS would suffice.
campervans
Yeh, definitely worth looking into, and comparing to a business connection
segmondy
Just do it. Lots of people are running home labs serving the internet, and we have been since the days of phone modem.
KomoD
I'd say you should consider renting a dedicated server or co-locating instead.
matt_s
This should be much higher in the comments.

If you're spending $1000/mo on hosting a SaaS you are very successful in my book. Margins on a SaaS could be around 80% which means you may have MRR of $5000. You should be building your SaaS out professionally and not using hobby/gaming PC parts. It might be a fun project but if you want growth, I'd advise focusing on the SaaS and not on the metal and plastic it runs on, outsource that.

Co-location costs are probably $100-300/month. So if you have an LLC you can purchase actual servers, write them off as a biz expense and let a colo provider deal with the power, networking, cooling, etc. of running a data center. They might have other services that you could leverage as you grow.

joshxyz
I'm really at awe on how he threw time instead of money on the problem.
jjice
I love the spirit of this! Like other have said, moving away from Heroku to a dedicated server that's already hosted would also be a win. Even on-demand AWS EC2 instances may be cheaper than your dynos.

I'd personally still lean towards a service where I could get a VM or dedicated box so I could keep internet access and power off my mind. That said, installing at a colo could be a good middleground for "bring your own" hardware, but power and internet included.

It's hard to say without knowing the nature of your SaaS because that can dictate risk tolerance and SLAs, but I'd run the costs on what the upside is vs the potential of an issue and what the cost of those issues are.

awsidiqi
We also moved off of Heroku, but instead of a home server, we moved to Hetzner (https://hetzner.com/) and threw Dokku on top. Significantly cheaper than Digital Ocean (ridiculously cheaper than Heroku) and has a good reputation on HN.

We opted for their VPS's in Ashburn, Virginia, but you can get dedicated servers in Europe as well. Depends where your customers are.

You can do this a lot quicker and more reliably than trying to run your own servers from home.

lulznews
What kind of workload are we talking about?
gregors
I like that you're planning on doing this. I think you'll have a good time reading these links.

* https://blog.codinghorror.com/the-scooter-computer/

* https://blog.codinghorror.com/the-cloud-is-just-someone-else...

Pay some attention to where he mentions "burning in your box"

I really think minipc's are the sweet spot. I run several and they are amazing for the cost. They run circles around cpu/memory/disk io you'll get at heroku or digital ocean. The cheaper smaller minipc's will run circles around any pi as well.

I run a couple of morefine m600 r9 7940hs ($520) with 1TB ssd and ($100) 64GB of ram ($200). If you look for deals you can occasionally score big (usually during holidays).

https://www.cpubenchmark.net/compare/3747vs5454/Intel-i7-107...

I'd say this, if you're serving "from home" I'd make sure you've got a fiber connection and that you're not behind a nat. It's actually not expensive to get fiber and a block of public ip addresses where I live, but you should check into that.

Also, I would absolutely buy a ups ....actually buy 2. One specifically for your networking set up and another for your box. Cheap insurance if you ask me. I'd honestly only start off maybe hosting your staging server. Figure out what the latencies are like, how you're going to do data backups, and OS updates, etc and all the other digital lawn mowing activities first.

Many small businesses used to do exactly what you 're doing, but there's risk involved. I do think the sweet spot is collocation, but if you're ok with the risks have at it. You'll learn a lot and many others have done it.

hiAndrewQuinn
You're probably better off moving to AWS or Azure, for all the reasons dsr points out.

You'll almost certainly still end up with significant cost savings. Plus learning how the cloud works at the base level is a valuable and transferable skill.

p0d
My last job was working for a growing saas company and this means security questionnaires and compliance. Your home server and environment wont tick many boxes.

I have been running my own small saas on Hostgator then Hetzner for the last 17 years. Go for cheap cloud servers. I currently pay £20 per month for £1000mrr

otmahthe
How much load do you experience with your server(s) ? What is the worst part to deal with ? Why specifically Hetzner ? (I've seen their offerring, it seems great and machine are quite powerful for the price)
p0d
My app doesn't drive enough concurrent traffic for me to worry about load.

The worst part is that things run so smoothly it makes me to lazy to do updates and check in as much as I should.

My main pull to Hetzner cloud followed having a good experience of their dedicated servers in my last job. Coupled with a fixed price for vms. I think the monthly traffic limit is 20tb. I'm nowhere near that

otmahthe
How much trafic do you have daily ?

I guess that you can also rent dedidacted servers or VPS and it will cut the bill by a lot, even on GCP or AWS, rent VMs with committed usage are not that expensive! It could also help with security and compliance.