Ask HN: Tips to get started on my own server

56 points by ctxc ↗ HN
What I want to do: get hands on experience hosting and maintaining a linux server (perf, sec, etc). I love the abstraction cloud services provided to enable me to build stuff without having to understand the nitty-gritty - but I think the knowledge will help. Rent a server on the cloud for about $10/mo if possible, build an application for personal use with everything residing on the server. Nothing critical.

-----

Why: I've always wanted to. I read some HN posts this week that have inspired me to finally make the leap. For eg,

- I read this and realised I don't know what any of these commands are (I've always used Windows), and it's time to start. https://www.brendangregg.com/blog/2024-03-24/linux-crisis-tools.html

- The SingleFile post. I already built one with Supabase that I use now, but I want to do one with the CLI, my own DB etc.

-----

Where I'm at: I have quite a bit of experience building both FE and BE for applications, mostly utilizing cloud services (serverless, hosted DBs etc). I've also hosted a few applications locally for personal use but not open to the internet, like Postgres and Nginx (all windows). Some devOps experience. Serious about security but no hands on experience with networking, I want to actually understand and reduce the attack surface and so on without just flipping a switch.

-----

What I'm asking for: Any suggestions how to go about it, resources, links, advice - please feel free to share your experiences as well. Thanks!

108 comments

[ 11.6 ms ] story [ 164 ms ] thread
Derek Sivers — Tech Independence

https://sive.rs/ti

Also what’s kinda cool is he encourages e-mailing him / reaching out . He’s pretty great

This is exactly what I wanted. I think I'm going to follow these steps and then branch out and experiment. Thanks a ton!
For sure !! ^-^ Happy to be able to help.

Definitely e-mail him if you have questions —

think he really enjoys the Hacker News crowd & is always game to be of service

My two primary tips:

1) Keep notes as you set it up. Include problems you encountered and what the solution was, develop lists of how to install and configure the various services so you can refer to them should you need to reinstall things in the future.

2) Do it one service at a time and get each service completely running before starting on the next. Your server isn't a single thing, it's a home for multiple things. Do each of those one at a time.

> Keep notes as you set it up.

This. I still have a 10 year old server setup file that I refer whenever I am setting up a new server.

In the off chance you don’t keep notes, I’ve found reconstructing what happened on the server from its history to be invaluable.

I once joined a company after they ran out of backend engineers and had to restore a server from a backup. Figuring out how to bring all the services up based on history was a fun ride.

Trouble shooting problems caused by someone else this way is also fun... "I see, they edited the nginx config ... let's check out what they did."
I'd recommend a note taking app, something like Obsidian, and create a page for each new service you set up. Copy the commands and add notes to this file.

Later, when you upgrade or modify the service, you can keep adding to this notes.

Also, copy any configuration files OFF the server and into you notes when you are done editing them.

I use obsidian, but brought my old hierarchical folder structure mentality into it. I now think I just should've created single files and let the auto-linking take care of the hierarchy.

What do you think?

I have a folder called operations/recipes and I put separate files in that folder. I have files like nginx, supervisor, etc. etc.
It may not be appropriate for a complete beginner, such as the OP, however, ideally setups should be documented by using a configuration management tool like Ansible, puppet, terraform, docker, etc. This way the setup is easily reproducible. The best notes are executable notes.
Yeah I think I'll try Ansible out. Maybe play around a bit with the bare instance and redo it with Ansible.
Thanks, I try to do it with my normal dev process as well. "preferably one unknown at a time" I call it, although I always end up wishing I had more notes. At least my blog would have more than 5 entries if I had half decent notes.

But never too late eh :P

One quick (but possibly retro) suggestion is to just host something small on your home network. Could be a raspberry pi. Getting the os and network to the point where you can host a simple website - perhaps with the aid of a dynamic DNS provider - could be a huge step.

What if you set a goal for yourself - that whenever someone clicks the user page for CXTC on HN, it contains a link to a web page on this server you have built.

Anything further (in terms of hardware, not-self hosting, etc) is incremental.

Completely agree. Use a raspberry pi or just run a VM on your desktop. Do your initial learning on something contained in your home network before you experiment (and potentially get compromised) on the internet.
I started with Linux by paying 5$/month for a Linode server, and I’ve never regretted it. I’m all for setting up your own hardware as well at some point, but it’s a different experience from having a server with 100% uptime to play with.
> host something small on your home network

I came to suggest the same thing. Another comment raised a good point about rpi vs a tiny x86—ARM may introduce some extra hurdles.

At the end of the day, especially when dealing with network configuration and security, you're going to make mistakes and get locked out. Having physical keyboard access can be a lifesaver and is how many of us "retro" people got started with linux administration

I wouldn't suggest hosting publicly open services from own local network. Without proper maintenance and experience, this is asking for trouble imo. VPSes are so cheap these days (far cheaper than 10$ if you don't need much performance), I'd recommend leaving local network out of the question.

When searching for cheap VPS, I usually peruse https://www.serverhunter.com/

I think it's a useful exercise in simple port forwarding, proxying (via nginx, for example), and certificate management. So long as you're not serving port 80 beyond redirecting to 443 and those are the only two ports being forwarded to the pi, it's really not that risky.
It might be "retro" in the sense that it isn't the primary way most businesses host their services, but I still don't think there is any better way to actually learn the ins and outs of networking and server management early on than to have non-obfuscated control and responsibility for everything you're doing.
I'm hosting my blog and my private Ghidra server on my Synology NAS. The NAS and the ISP router are already always-on, so it "only" costs me ~20€/year for a domain name.

Even accounting for the electricity and network I'm going to use anyways, it's quite cheap.

Thanks for the great suggestion! I think I'll start with the hosted server and then move to this.

I've been burnt too many times (by myself), buying domain names and arduinos and cool things I never got around to using .-.

Digital Ocean droplets can be created within your price range with a very vanilla version of Debian installed. My personal website runs off one.

I've used AWS a lot for larger projects but they do a lot more to create vendor lock-in which breaks the idea of a "Linux server", i.e. it's easy to end up inadvertently depending on Amazon tools.

Running your own physical server is also a valuable experience, but it's a significantly larger commitment which is less reward-per-effort in my opinion.

If you use a Mac and just want to mess around with linux try something like Orbstack(https://orbstack.dev/) to start up VMs and mess around. The benefit of this is you're going to break things a bunch as you get started. Going from there I'd start looking automating the deployment of the various components the 'old fashioned' way aka writing shell scripts/using SSH. Once you do that then go to using things like Ansible or Terraform etc.
Windows, but yup - I'll start with the raw stuff and move to Ansible.

Till a while back, ssh sounded scary though xD

My experience is I did a systems administration internship and i had to play with all the services, it taught me a ton about tools and layers below where I usually operate (Go APIs and backend services). If you're just wanting to get skills on system administration and linux. I'd start with either 1) an old machine you have lying around (or pluck one from a recycling center) or 2) a virtual machine, and 3) once you have confidence on a single box, start to use multiple to make clusters/replica sets of things like multiple nodes of HTTP server, or multiple nodes of a mongodb, etcd or other distributed systems. Try doing work on them and pulling the plug on one mid execution to see what happens.

From there try to play with a bunch of the following things: (OTOH, no particular order)

setup a domain name server like bind (<something>.localhost?), setting up an SMTP service, setup an IMAP/POP3 service, set up a http server like nginx, lighttp, apache, try communicating to the http service over telnet, setup a https certificate with lets encrypt, try doing the same telnet to https and get it piped through gnutls, setup nightly backups to an external HD (try rsync), setup a spam filter for your SMTP server, try to run excalidraw on your http service, build an api that sits behind your HTTP(s). use tcpdump and wireshark to inspect network traffic.

Most of that should be google-able :)

Edit: MOAR

fail2ban, ssh-server, iptables firewall, irc server, sftp host, kubernetes cluster of virtual machines, hadoop/spark for a map reduce workload, check out https://www.cncf.io/projects/ and start to learn to build with those projects/abstractions.

Learning to make snapshots at different steps so you can restore to them and not have to build from scratch is a big help too.
That's a good list, thank you! Will keep the ~Google~ duck overlords busy :)
Make and host a simple web app:

Django(Python) - gunicorn - nginx

You could run it on a raspberry pi for no ongoing cost

You can find super cheap VPS offers on lowendtalk.com. I wouldn't trust many of them for production, but even the cheapest one should be good for learning. Also, Hetzner is always a reliable low-cost option too.
Dig up an old laptop, put a beginner OS on it (like Ubuntu). Pick something you want to do on it - like a shared network folder. Then figure out how to build it.

If you need a full server to do it - that's a toy that'll get in your way.

Oracle cloud has a great always free tier. You can get a 4 core 24 GB ARM server which is awesome. Of course, it is obvious ARM so while support for that is generally pretty good there may be some quirks.
They ban people for no reason, they harass people with "we would like to talk to you" emails. They refuse to delete your data and name email address and so on after you've been banned.

They also require their own 2fa app instead of other common apps e.g. Google authenticator. And when you delete it, after you've been banned for no reason, you can't use it to request deletion of your data.

Do not go there. Oracle "free" tier isn't free. It's a ploy to grab your data.

You have been warned.

Honestly I'm more surprised that there still exist people willingly using anything from Oracle or advise doing it. Oracle is The Asshole.
I don't mind having a free server from them! I ignored a few sales emails but otherwise it's been drama-free.

If you're looking for a place to experiment I don't see any better deal.

I'd suggest setting up a raspberry pi and build your own service locally hosted at home!
Provisioning is something that may be out of the picture you described, Terraform, Ansible, templates, whatever, more than creating that server try to recreate it after destroying it (and backups while at that). Monitoring using your own tools instead of (or along with) your provider tools will give you a view of not just when it is working wrong, but also when is working right. Security is another thing that goes both for the cloud and your own server, but for long running instances firewalls and vulnerabilities in what is exposed gets more urgent.

Regarding performance, if you want to dig in that topic, don't wait for a crisis for using those tools, try to understand how the system run, even if not under high load. Gregg have more tools (https://www.brendangregg.com/linuxperf.html) and a few great books.

Yup, I got to focus on monitoring as well! Will check the tools out
>Rent a server on the cloud for about $10/mo if possible

It can be cheaper to buy your own "mini" server (e.g. a small N100 ITX computer) to use on your home network. You'll get more RAM and more disk space (e.g. 16GB ram 512GB disk) to play with compared to datacenter rental prices cumulatively adding up over a year.[1]

Unless the specific aspect of renting a public-facing server is part of your self-learning curriculum and/or you also need to access it from outside your home without NAT port-forwarding or Cloudflare reverse-proxy tunneling, you can self-host your playground server at home.

[1] compare the rental costs and cross-reference GB of RAM and GB of disk they include for the monthly price range you want to pay.

https://www.digitalocean.com/pricing/droplets

https://www.linode.com/pricing/#compute-shared

look into the selfhosting hobbyist scene (eg r/selfhosted) and find some cool stuff you want to run for yourself. then either run it on your lan on a spare machine or get a cheap vps and set it up in the cloud. doing stuff with docker is the easiest imo but setting stuff up by hand by installing packages, modifying config files etc is useful for getting your hands dirty. interrogate chatgpt about anything you don't understand.

the bash `history` command is your friend.

Ah, forgot about reddit. Will check that one out!
- have a firewall and configure services on a needed basis.

- move sshd to other than port 22, there will still be brute forcers but it will keep the spam down in the log. You can have .ssh/config with your hostname and that set port, so you don't always have to ssh hostname -p 23456.

- speaking of sshd, fail2ban is useful

- contabo.com has cheap vps

Apart from that, what's there to say. I can't think of anything right now.

> "I read this and realised I don't know what any of these commands are (I've always used Windows), and it's time to start."

I'm a great admirer of Brendan Gregg but I have to say that 1) that's trying to run before you can walk; those tools require fairly advanced Linux knowledge to operate properly and you say that you've only used Windows, and 2) if you're primarily interested in the performance analysis aspect, analogous performance measurement tools exist in Windows, like Windows' built in performance counters and PerfMon.

If you're unfamiliar with Linux and want to get to a point where you understand the tools Brendan Gregg mentioned, I'll say something similar to what others have said: get an inexpensive used 1-liter PC¹ at home, load Ubuntu or some other popular Linux distribution, and start with Nemeth's "UNIX and Linux System Administration Handbook" to learn your way around the system. That will give you the context you need to be able to start to understand how performance analysis tools are used. After that, you can move to the cloud whenever you feel you're ready.

¹ The description at https://www.servethehome.com/introducing-project-tinyminimic... explains what they are and why they are a good value. I wouldn't recommend the Raspberry Pi because not all diagnostic tools are available on ARM and a PC is no more difficult to re-image if you make a mistake than a Raspberry Pi.

That was a good article! I saw the handbook recommendation on a other HN post as well, will take a look. Thanks!
One great thing about setting up your own server is to do it faster each time. Deployment, hardening, running services.

Get used to your terminal, use SSH, understand how firewalls work and run a couple of services such as nginx, fail2ban etc.

Once you have a hand of it try to do it using Ansible. It will be a valuable skill if you decide to provision, for example, a swarm of servers for a cluster. And it will give you reproducibility and a trail of steps that you will ensure get executed for each server you deploy.

It hasn't been suggested yet so here it is: I found plugging a raspberry or a refurbished/second-hand NUC on my home LAN made some things click faster than with VMs on computer or a single remote server. After a while, it slowed down progresses though, but it depends on what you actually do.

You really need to understand the basics of IP and networks though, so get that right first.

Try to run your application for a year, and when you run into problems don’t just fix them, find solutions to keep it from happening ever again.

Stick to some LTS linux distribution so you don’t have to think about that stuff too much, it’s annoying otherwise.

When you’re ready for hard mode start doing it on a raspberry pi on like a 0.5MB/s link, and add random reboots by pulling the plug.

Once all that’s done you’re probably close enough to found a device management company.

Haha, I aim to get my website running on it!
I switched from DigitalOcean to Hetzner for my cloud servers. They have a nice web firewall, that you could use. I use it to prevent access on some ports, for example on Port 22...

But wait, how can you connect to Port 22 then? I use Tailscale as a VPN for this. I installed Tailscale on my local computer and my servers, so I can use the VPN to connect to this port. The less open ports, the better.

What else... Use fail2ban or CrowdSec for banning IPs that probe your server. Don't use passwords for authentication. Don't use root, use another user. ... stuff you maybe already heard of or know. Here is a link to get started: https://blog.codelitt.com/my-first-10-minutes-on-a-server-pr...

If you knew this already, sorry, I wasn't sure where to start.

Maybe use a Hoster that provides 'Snapshots'. So you can safely play around and if something happens, just revert to your last snapshot, so you don't loose your previous work.

If you spin up a server, maybe use cloud config. You can use it to 'bootstrap' your server and let it install your main tools while it gets provisioned.

It looks something like this (HackerNews removes my line breaks...): ####

#cloud-config for installing fail2ban

package_upgrade: true

packages:

  - fail2ban
runcmd:

  - [ systemctl, enable, fail2ban ]

  - [ systemctl, start, fail2ban ]
####

If you start and mess up something, it's handy if you have a cloud init to start a new server and don't have to install the basic stuff again ;-)

I didn't know that - so thank you! I'll start with the link :D
Sounds like the easiest thing to do would be install Linux to a VM locally and configure the same services you've hosted in Windows, but now in Linux. Test it out from your client to make sure everything that worked on Windows is also correct on the new setup.
Buy a Raspberry Pi. Get it on your network, and get used to SSH'ing into it. You can learn a lot with a Raspberry Pi and if you break it you can just redo the SD card for the OS.

I also recommend finding an old laptop and installing Ubuntu on it, you would be surprised how often doing just this brings new life to an old laptop. I usually get it a new SSD for $50 and it feels like a brand new laptop whenever I do this.

If you really want a VPS then OVH, DigitalOcean or even BuyVM offer cheap options. BuyVM will get you a KVM slice for $24 per year.

Yup, this is what a lot of recommendations say...will test the waters and go the hardware way!
Depending on what you want to specifically learn, I'd start with the easier distributions, no need to waste hours unable to install or use an OS. Then if you want to "REALLY LEARN" I'd suggest trying out something like Slackware first before jumping straight into the Gentoo or Arch Linux bandwagon. What you'd learn from Slackware is things like partitioning during installation, and how useful package managers can be ;)

Also some commands you want to always use:

man is #1 cause it will show you documentation on any other command and how to use them

almost every program has a -h or --help that tells you how to use them as well.

Don't do one-off setups and quick fixes. The entire knowledge will be lost on the next instance failure. Launch and provision the instances automatically with well maintained scripts. Start with cheap VPS to initially avoid dealing with entire domain of problems related to physical servers.
Some tips:

1. Digital Ocean offer small VPS for $5 per month. That's a 50% saving right away!

2. Stick with Ubuntu in the beginning. It's not the best, but it's 100% good enough and has so much support and tutorials out there.

3. If you have a small VPS with not much RAM, definitely set up a swapfile. It gives you virtual RAM for doing RAM heavy things on a small VPS.

4. Use the virtual firewalls offered by your host rather than the server firewalls in the beginning. If you mess up a server firewall you may have to get your host to reset it for you. If you mess up a virtual firewall you can amend it through a web UI and get back to doing things quickly.

5. Learn to read man pages and log files. Between the two you can figure out how to do stuff, and then figure out why it isn't working correctly.

6. In terms of security, use a recent distro, use a firewall close everything you don't need, use SSH keys, and set up secure passwords for everything else, and you will avoid a lot of problems.

7. Keep an eye on resources, programs like top, uptime, free, df, and du will allow you to see what's using up CPU, RAM, or disk space.

8. Learn a relational database. MySQL or Postgres are good choices. This skill will keep you employed for years, almost every business uses a relational database in one way or another!

9. Have fun :)

Curious why you say Ubuntu is not the best. What would you consider better?
Not OP, but ubuntu is among the best documented an supported, which is critical for beginners.

What may not be good about Ubuntu for some is that it's too bloated and not difficult to learn for someone who wants to hang their hat on that.

Moving from Ubuntu to another Debian installation, including debian is no big deal.

Personally I think debian is the best server OS to learn. It's sufficiently similar to ubuntu that you can get away with either one really, they aren't very different as ubuntu is based on debian.
I'd recommend Debian as the uncontroversial Linux server distro.

But for someone who might have some Ubuntu experience and little other Linux experience, running an Ubuntu server might lead to fewer surprises. Ubuntus do ship with things that Debians don't have. I can't name any, since I don't use Ubuntu, but they're the things about which you either say "how nice!" or "how dull!"

I think it's a question of how sharp the tool is, and how 'idiot-proof' it's been made. Ubuntu is as idiot-proof as Linuxes go, but the more knowledgeable you get the more you find use for a sharper tool that fits your domain requirements better.

Debian is molded to the server domain pretty closely; Red Hat / Rocky is ideal in regulatory environments; Alpine when the priority is to be lightweight and reliable; I run Void on my personal machines because it's the most BSD of the Linuxes and stays out of my way when I need to do weird stuff.

If you're a beginner, you can append 'ubuntu' to your google searches instead of 'linux' and get an answer that makes sense. You'll know when to move on from ubuntu when you have to search 'foo bar linux' to get a usable answer.

I tend to append archlinux to google searches to get quality answers from ArchLinux Wiki.
I will add 6.1 use containers and keep you VPS clean.

P.s. as an alternative to ubuntu you could check Alpine Linux (for both root vps and containers)

Good luck!

Thanks! Especially for the swapfile, hadn't heard of that one before. And your comment history makes me want to read up about ansible as well :D
Ubuntu Server is superb! You can install it with one click in AWS Lightsail, configured securely with or without apps, with a public IP.
You might like the t-series servers from AMZN

https://aws.amazon.com/ec2/instance-types/t2/

the t2.large costs about $10 a month and is particularly good for the bursty loads you'd expect for a lab server. You can get a t2.micro on the free tier as well which is a pretty weak machine. I would watch out because I once ran OpenVPN on a t2 instance that had way too little RAM and it went swap crazy and ran up a $200/month I/O bill. (For the life of me I cannot understand why AMZN doesn't support a branded VPN server that "just works")

Note Azure has the B-series which is similar to the AMZN T-series

https://azure.microsoft.com/en-us/pricing/details/virtual-ma...

and my understanding is that this is about as good.

TIL ram swap! I always thought processes would get shut down when ram is full...thanks!