Ask HN: How can a total beginner start with self-hosting?
Gradually, patiently, persistently, over the past ten years and more, I moved from Windows and Mac to all FOSS apps and then full Linux. Doing the same with my phone. Total success. Independence and self-reliance.
In short it’s all about control, privacy, and security, in that order. And: it’s a long term process that requires a commitment.
I understand desktop Linux (Ubuntu/Pop!_OS) well enough to get myself out of trouble when I mess up or an update breaks. But I have no clue about networking, and I don’t know where to start.
Syncthing keeps a handful of my important directories of user-files synced quite reliably.
I deleted my Google account years ago. But I’m still in iCloud and iOS for all the photos. Highly recommend Fastmail incidentally.
I have a small cheap Linode VPS (doing nothing right now), a Mullvad client on all my devices, Tailscale on all my devices (doing nothing because I don’t understand what it can do), and a Synology NAS in the closet with the modem/router (none of which I understand).
I want to:
- host my own photos and get out of Apple.
- host my own bare git repos and not rely on GitHub.
- host my own BitWarden server.
- host my own Tail-/Headscale (whatever the noun is).
- follow up on ideas that pop up after I comprehend networking.
I can HERPaDERP install packages on client and server, and copypasta configs I don’t understand. Where do I go to understand?
193 comments
[ 2.7 ms ] story [ 254 ms ] threadPlay and experiment. That's how I started, and as long as you have lots of off-line backups, you can get pretty far.
Tailscale makes two or more computers look like they're on the same network (simplification).
Later you can decide to keep things on your little virtual host on your home IP (depends on your connection and requirements) or migrate to a VPS at Linode, etc. I like having it at home with me, but that's just me.
Once I understood somewhat what ESXi was doing, it was easier to learn proxmox or Xen or whatever.
I'm currently using Cockpit Project (and quite happy with it).
I have an old computer that I connected to my router, and I’m able to ssh into it and do stuff.
It’s an old intel quad core with 16gb ram, and has a 1TB SSD. More than capable enough to handle a bit of workload. It runs Ubuntu and I’m using it to run backends for apps as I develop them.
I don't see much downside to sticking with Tailscale indefinitely, what are your reasons?
> I don't see much downside to sticking with Tailscale indefinitely, what are your reasons?
Tailscale and Bitwarden both fall into the same category: they seem like good actors, they have generous free tiers, and if I come to rely on them at scales that aren't free I would be happy to pay. However, I want to understand them enough to be able to know that I could host them myself if god forbid they go out of business, are subject to attack, or whatever.
As I've succeeded at on the desktop, the hobbyist gratification is both the fun of tinkering mixed with the confidence that if it need it I have packed my own parachute.
Some starting points
- photos: NextCloud
- git: Gitea
- BitWarden: Vaultwarden (even if you deploy this locally you want a SSL certificate as clients will refuse to connect otherwise)
I'd suggest using official docker images to get started as there’s plenty documentation available for all projects and experimenting is a bit easier when you can simply dispose a container without having to worry what’ll happen to your host OS.
As long as you run services locally on your Synology (assuming it supports docker) and don’t expose them to the Internet I’d encourage you to „just give it a try“.
Just don’t immediately start to rely on the services and run a dual strategy (NextCloud and iCloud photos for example) till you updated your container once or twice and feel comfortable troubleshooting issues with your stack. Nothing is more discouraging than having a service you need „right now“ being down and no idea how to get it back up.
It’ll be a long, fun journey. Good luck!
If you don't mind horrible experience viewing photos/videos via nextcloud, go on. In my case this was unusable. Thumbnails not pregenerated even after trying (Yeah, didn't spend whole day on that issue) and generates on the fly. So viewing larger directory is... rubbish. Videos don't play as nice not to say they don't even have thumbnails. Feels like "guest book" from 2000 - no features that auto organizes stuff - just a directory with photos and you're on your own with unorganized mess of photos.
How great was HN when it suggested me https://photoprism.app/ - and it really just works! Nice, performant, featureful, yet feels lightweight. Finally I can view my photos.
I still use nextcloud just for sync and photoprism just has directory mounted as readonly. Still, sync from phone feels heavy along with "failed to sync" errors and just hangs doing nothing... I long to try out syncthing - but then I loose web access to documents which.. maybe someone can suggest some frontend for that?
Someone also suggested https://photostructure.com/ - it looks decent, haven't tried out.
Nextcloud will soon be at version 25, which will also be named Nextcloud Hub 3. Frank Karlitchek talks about this during the Q&A, about an hour and a half into the video linked below.
However there are major improvements coming in that new version, specifically to the Photos app[1]
[1] https://www.youtube.com/watch?v=dhJXZzqsv8A&t=1103
I've gone through some of it and it seems like a decent primer on where to start, but I'm not sure that it has all the required info that OP would want.
"Why gitea and not bare repos?"
- Better repo, user, and access management, all from a browser.
- LFS support.
- Being able to browse code in... well, a browser, is really nice.
- GH-like workflows generally, if you want that.
You can have it use sqlite for the DB to make it extremely easy to manage and backup and such. I'd expect that to be fine up to at least 50 moderately-active users, and maybe much higher.
https://www.chezmoi.io/
How do I do all this while exposing it to the internet? I want to host stuff for my friends and family without putting them on a VPN to my house.
[1]: https://hub.docker.com/r/cloudflare/cloudflared
- CompTIA Network+
- Linux Foundation Certified IT Associate
For extra credit, pass the Linux Foundation Kubernetes certs, get a AWS cert, pass the Offensive Security PEN-200 cert, or take any of the GIAC certs. These won't make you competent, but they'll provide a baseline that you can quickly attain which will get you started.
After those, maybe consider project-based learning.
- Install Arch Linux
- Install Linux from Scratch
- Learn to use QubesOS and make your own OS templates/ISO.
I'm certain others here will say certs are a waste. I do not agree. They are a way for people who don't have enough context to quickly build that context.
Good luck!
In your case that might be migrating your photos off iCloud. I found the awesome-selfhosted[1] list to be excellent for trying out different products that match the size of the VPS you've got or maybe you just want to put that onto your local Synology NAS if you don't need your whole photo roll on the go.
Self hosted BitWarden is also another good starting point with the very lightweight vaultwarden[2] just make sure you always know where your vault is stored on your server and make backups.
While it's a long road it doesn't need to consume your life daily but it still requires you to keep up with all the things any sysadmin needs to handle like monthly patching, monitoring the logs for sustained abuse and break in attempts.
Subreddits /r/selfhosted and /r/homelab are also great places to have a browse.
[1] https://github.com/awesome-selfhosted/awesome-selfhosted#pho...
[2] https://github.com/dani-garcia/vaultwarden
My path was to use Linux distributions that are well-documented that you can assemble piece-wise. Examples include Slackware, Debian, and Arch. By understanding the pieces you’ll come to understand networking better, and you’ll better understand how to help yourself.
That’s just one path though, certainly there are others. Just look at how far you’ve come, and realize that with time you’ll pick up more.
I can vouch for git repositories being easy to host on a VPS. I use a private git repo as my daily backup tool for my documents. A public one should be easy too. Access management for particular users, I'm not so sure about.
Slap on a distro, and you're off to the races. Checkout /r/homelab and /r/selfhosted on reddit. You'll probably want to read about DNS and local networking (DNS & Bind is a good book).
You'll understand by doing things. Don't blindly copy-paste configs. Spend some time to figure out what they're doing and type every line in manually.
Ad blocking for your phone? VPN for work? Self hosted email? Retro gaming? Figure out what you want most and jump into that instead of trying to get everything all at once as it can be overwhelming to consider every system instead of taking one step at a time.
I have a very beefy desktop for the first time in over a decade (maxed out Mac Studio) and I am wondering is there any real downside to having these services run on it?
I've found a raspberrypi/orangepi to work just fine and it doesn't matter if the main computer is on, I'm upgrading to a new system, or traveling between work (laptops). It just sits in the closet and does it's thing using a tiny amount of electricity.
My desktop gets shut down, rebooted, crashed, wiped and reloaded, and otherwise abused as I get thoughts.
My home server has been the same (don't worry I run updates) for a couple years, chugging along, sipping about 1/4 the power my desktop would take.
In the beginning of your self hosting journey, the opposite might be true. You might want to try 300 things, and that's hard to do if you're trying to get work done on your desktop.
1. My 12-year old, former top-of-the-line laptop is much faster than my 3rd-gen Pi when periodically ingesting and indexing JSON into a PostgeSQL db, I suspect slow IO and limited memory are to blame.
WoW (wake on wife), worked pretty well from the office, but I seem to get a little bit of latency and packet loss. Now I work from home, so I just get up and hit the power button on the more powerful machines.
My shock with compute power was when I got the Pi 4. It was faster than the AMD e350 I was using for backups except for disk IO. The E350 was not a powerful machine new, but I didn't realize how far we had come.
A USB 3 enclosure brought it to a level that was good enough the entire mini ITX build was pointless to keep around.
A pi/old computer is a physical box you can put in a corner and you can be sure the physical network works if it is connected.
It's reasonable to assume that the 115 W on a Mac Studio will never be hit on a workload that could be supported by a RPI...so you're looking at an extra $10-20/yr to operate the Mac Studio.
The biggest issue is just the upfront cost, since of course the Mac Studio is proportionally expensive to its capability.
E.g. you can't restart your desktop while your roommate is watching a Plex movie. Or your VPN stops working because your roommate's cat walked on your keyboard while you were on vacation.
I like the VM rather than running it directly on the Mac because it's trivial to copy the VM to another computer when I want to change hardware. Obviously, your services will be unavailable whenever the host machine restarts.
Especially for DNS. Like, your system reboots for an OS update and now the rest of your the devices on your network have connectivity issues for an hour or so. Or you have a hardware issue and now you lack both the infra and the workstation so getting things up and running again becomes triply annoying.
Definitely still an amateur in my networking knowledge but I've learned a ton over the past year.
Also, if you are into Docker, I love the images hosted by these guys. https://www.linuxserver.io/
I would add two things to the mix here - the first is there are a bunch of blogs who cater to selfhosting as well as on YouTube. The ones on YouTube are much easier to find. I'd say one of the more active bloggers though is Jeremy who runs Noted [0].
The other thing I'd take into consideration is if you want to manage your selfhosting, and if not there are options there as well. One popular option as of late is Umbrel [1] and there's also some like Sandstorm [2] that have been around longer. Yunohost [3] also seems to have some traction in this type of self-hosting realm.
[0] https://noted.lol/ [1] https://umbrel.com/ [2] https://sandstorm.io/ [3] https://yunohost.org/
https://www.reddit.com/r/linuxupskillchallenge?utm_medium=an...
It's a beginner course on Linux administration - not networking. It will give you enough knowledge to understand and manage a server. It's free, and starts on the first Monday of each month (you can also do it self paced if you like).
Most of application deployment is little more than reading the docs and tuning the configuration to your needs. From what I read, I think you've got enough knowledge to get that stuff running on your servers. You can probably get a lot more out of learning about the underlying concepts.
For your own photos and cloud: I use Seafile, have used Nextcloud, and alternatives exist. Quite easy to set up, but with the ability to go deep into Modern (TM) Cloud (C) backends if you want.
For your Bitwarden setup: Vaultwarden is a lot easier on resources and has pretty much all the features you need. Also quite easy to set up.
For your tailscale setup: there's a guide for the server (https://github.com/juanfont/headscale/blob/main/docs/running...) and you can find more guides for the clients.
For your Git setup: Git works over simple SSH. If you can SSH into your server, you can host a git repository. If you want more (a nice web GUI) then Gitea or Gitlab can also be run on your server.
Things I recommend reading into if your knowledge about them is spotty (find guides or book recommendations):
- Networking (ARP, IPv4, IPv6, TCP, UDP, DNS, mDNS, maybe PPPoE, and other such abbreviations). This is a lot of reading. You can also try to get started with this stuff without reading into it (it's how I learned!) and have a terribly frustrated time by overlooking obvious mistakes and easy solutions, but I don't recommend that.
- SystemD services. People use Docker to solve a lot of daemon problems but good ol' systemd can do a huge part of that! I run most of my services in systemd rather than some kind of container setup because I don't want to have to deal with Docker and its many friends and dependencies whenever I'm trying to resolve a problem and so far it works great.
- Reverse proxies, if you're running multiple services on a single server with subdomains or subpaths; learn about nginx/caddy/apache2/whatever server you prefer and how to set up proxying. Along the way you will break stuff and learn new things with every error message or unexpected routing error you encounter!
- Firewalls; firewalld and ufw are nice ways to get started, nftables/iptables for the underlying stuff. It's not hard, per se, but it can get complicated fast. Maybe mess with the Windows firewall as well just for fun.
- Set up IPv6 if you don't have it already. This would allow you to do some more networking stuff and prepare you better for the future, because corporate networking people seem to be grumpy and annoyed at the thought of one day needing to enable a protocol from the 90s. If your ISP only does IPv4, https://ipv6.he.net/ will get you an IPv6 subnet for free and if you do all of their quizzes they'll even send you a free shirt!
- Along the way, you will (or should, at least) learn to use Wireshark and friends. Incredibly overwhelming at first but with some knowledge about networks you'll get the hang of it by setting up the right filters.
This is the kind of reference I was looking for. Anybody else have a favorite. Maybe more accessible? "For Dummies?" :-)
Thanks for all these concrete suggestions.
EDIT: "Free t-shirt"?! AWESOME :-)
Most of those chapters are probably not relevant to you. All the wireless stuff is nice if you want to dig into setting up your own radio network but you can stick to the internet stuff. Not even everything in there may be relevant to your interests, I've forgotten most of the network/routing graph discovery stuff myself because it's not really relevant unless you're planning on running a corporate network or an ISP. I recommend just giving it a good go, and skipping the parts you probably won't ever need. You can always read them later if a later chapter refers to them and you need to understand after all.
As for Linux stuff, last month's Humble Bundle had a bunch of Linux related books: https://web.archive.org/web/20220913000217/https://www.humbl...
You can probably find people talking about this bundle online and find out what books/alternatives they recommend. There were a few Reddit threads about this at the very least, maybe a HN thread or two as well.
I believe it's made by Luke Smith, with content uploaded by other people.
My best answer is: find a mentor.
Someone you can repeatedly ask for detailed pointers from as you get stuck. This could be a colleague, an IRC/Discord friend or even someone on Twitter that you have bonded with.
I have been mentoring people close to me on computers and Linux since I was about 13 years old and am now 39. And it has been a real blessing, since you learn a lot by being forced to explain what you already know.
As a teenager I didn’t think of this as mentoring of course. But I wad very lucky to have had my 3 years older brother as computing mentor, which gave me a great head start compared to my peers.
Not knowing exactly where you or others reading this comment are currently getting stuck, here are a few random pointers:
netstat -a -n -l -p
ls -la /proc
man mdadm
iptables -L -n
rsync -a -e ssh myfolder user@host:
And reading Beij’s (?) tutorial on TCP socket programming if you are an aspiring C programmer.
Thanks especially for the concrete list of topics to bork around with.
As others have mentioned, proxmox, Unraid, and/or TrueNAS are great if you have unused/extra hardware sitting around. Personally I have a box for Proxmox VM's, and an Unraid server for storage and several docker containers i use regularly. I'm still very cloud dependant for the convenience factor, but this should help give you some direction.
There are also communities on reddit like /r/selfhosted and /r/DataHoarder/ that you might want to check out.
You should also run fail2ban on everything.
I'll follow these pointers for sure.
Oh and very much avoid random tutorials on the internet. As in, go for official source and use these tutorial only to connect the dots. The reason is that there is huge amounts of really bad advice on the internet and a lot of the tutorials only work in very specific situations (versions, OSs, etc.). Official documentation tends to be a lot better, and it's a good idea to choose software that provides good documentation.
Also make sure you do it one step at a time. You want to give things time to know what failure cases it might have. This prevents you from situations where everything "crashes and burns", because there is an update.