Ask HN: What do you self-host?
I know this is has been posted before but that was a few years ago so I wanted to restart the discussion, as I love hearing about what people host at home.
I am currently running an Unraid server with some docker containers, here are a few of them: Plex, Radarr, Sonarr, Ombi, NZBGet, Bitwarden, Storj, Hyrda, Nextcloud, NginxProxyManager, Unifi, Pihole, OpenVPN, InfluxDB, Grafana.
346 comments
[ 5.0 ms ] story [ 306 ms ] threadNow I only host my own project: http://billion.dev.losttech.software:2095/
Also regular Windows file sharing which I use for media server and backups.
Though I'd like to expand that. Maybe a hosted GitLab.
What do you feed into Grafana?
I have a home server + some raspberry pis lying around that I want to start using.
* It's a target for my rsync backups for all my client systems (most critical use); Docker TIG stack (Telegraf, InfluxDB, Grafana) which monitors my rackmount APC UPS, my Ubiquiti network hardware, Docker, and just general system stats; Docker Plex; Docker Transmission w/VPN; Docker Unifi; A custom network monitor I built that just pings/netcats certain internal and external hosts (not used too seriously but it comes in handy); and finally a neglected Minecraft server.
I went for low power consumption since it's an always-on device and power comes at a premium here + fanless. I highly suggest the NUC as it's a highly capable device and with plenty of power if upgraded a bit!
Man, at my last job in a large enterprise, I WISH they were running fingerd. Would have made for some pretty cool, lightweight integrations.
I also host Aether P2P (https://getaether.net) on a Raspberry Pi-like device, so it helps the P2P network. But I’m biased on that last one, it’s my own software.
I still have the email domain, because it's easier to run it forever than migrate all the things you signed up for. But actually running my own email is too much of an obligation and need to keep up on all the anti spam measures.
You don't want less tested web app to expose some security hole for someone to start snooping on your traffic toward BitWarden after SSL termination.
If you don't want an extra box at home, you can always get a $5/mo cloud instance for public stuff, where you don't have to worry about increased electricity bill from DDoS having CPU spiked or choking your home network.
I am unhappy with the complexity of Mayan EDMS. I'm debating moving to Paperless. All I want is a digital file system that 1) looks at directories and automatically handles files 2) has user permissions/personal files so I can let my family use it 3) has a web form for uploads.
I am planning to change gitea to sourcehut- the git service as well as builds.
Any ideas for things a raspberry pi 3 & 4 could be useful for?
I have a second raspberry pi running a version of Kali Linux. I only hack my own stuff for learning.
Once upon a time I ran a public facing website and quake server, and published player stats. No time these days for much play.
Incoming mail points directly to an RPi at home on dsl... Postfix + Dovecot IMAP. It's externally accessible, my dedicated server does the dynamic dns to point to the RPi; the domain MX points to that. Outgoing mail forwards through the dedicated server, which has an IP with good reputation and DKIM.
This gets me a nice result that my current and historical email is delivered directly to, and stays at, home, and my outgoing mail is still universally accepted. There's no dependency on google or github. There's no virtualization, no docker, no containers, just Linux on the server and on the rpi to keep up to date. It uses OS packages for everything so it stays up to date with security updates.
• Apache: hosting a few websites and a personal (private) wiki.
• Transmission: well, as an always-on torrent client. Usually I add a torrent here, wait for it to download and then transfer it via SFTP to my laptop.
• Gitea: mostly to mirror third party repos I need or find useful.
• Wireguard: as a VPN server for all my devices and VPS, mostly so I don't need to expose SSH to the internet. Was really easy to setup and it's been painless so far.
* MinIO: for access to my storage over the S3 API, I use it with restic for device backups and to share files with friends and family
* CoreDNS: DNS cache with blacklisted domains (like Pihole), gives DNS-over-TLS to the home network and to my phone when I'm outside
* A backup of my S3-hosted sites, just in case (bejarano.io, blog.bejarano.io, mta-sts.bejarano.io and prefers-color-scheme.bejarano.io)
* https://ideas.bejarano.io, a simple "pick-one-at-random" site for 20,000 startup ideas (https://news.ycombinator.com/item?id=21112345)
* MediaWiki instance for systems administration stuff
* An internal (only accessible from my home network) picture gallery for family pictures
* TeamSpeak server
* Cron jobs: dynamic DNS, updating the domain blacklist nightly, recursively checking my websites for broken links, keeping an eye on any new release of a bunch of software packages I use
* Prometheus stack + a bunch of exporters for all the stuff above
* IPsec/L2TP VPN for remote access to internal services (picture gallery and Prometheus)
* And a bunch of internal Kubernetes stuff for monitoring and such
I still have to figure out log aggregation (probably going to use fluentd), I want to add some web-based automation framework like NodeRED or n8n.io for random stuff. I'd also like to host some password manager but I still have to study that.
I also plan on rewriting wormhol.org into supporting any S3 backend, so that I can bind it's storage with MinIO.
And finally, I'd like to move off single-disk storage and get a decent RAID solution to provide NFS for my cluster, as well as a couple more nodes to add redundancy and more compute.
Edit: formatting.
FYI: the control plane takes about 150m (milli-cpu) and ~1.5GiB of memory in a host with my specs.
The thing is, I don't use Kubernetes for convenience or because I need it, I use to learn it.
I was just fine with Docker Swarm before switching, but I wanted to learn Kubernetes as a valuable skill, and I know no better way of learning something than using it every day.
And the thing about Kubernetes distros is that they usually all apply a new layer of "turning Kubernetes' complexity into a turn-key process", and I don't want that.
If you know the ins and outs of K8s, sure, use any distro you like, but if you want to learn something, better learn the fundamentals first. It's like learning Linux's internals instead of learning how Ubuntu is, one applies to a single distro and the other will apply for every distro ever.
k3s is not very far from the fundamentals. It's really just "one binary" instead of many for the space savings/ simple deployment.
That said, consider Kubernetes in Action by Manning. I'm about 75% done now, was a great help, and I'm continuing with k3s after doing it.
I bought Kubernetes Up & Running a year ago, I was disappointed to see it is a very over-the-top view, without getting into details.
I skimmed over Kubernetes in Action a couple months ago. Nothing really catched my eye either.
The last one I read was Kubernetes Security by Liz Rize. Either there's not that much to securing Kubernetes or the book is very introductory too.
The only parts of K8s I don't know a lot about are storage (haven't got past the NFS driver yet), CRDs and distributions like OpenShift. But in the same way I'm lacking storage expertise outside of Kubernetes.
I could set up a demo if you want to.
It's a cheap Flask app that scans a given "library" directory for "album" subdirectories, which contain the pictures you want to display.
It has a big issue with image size (16 images per page, my phone takes 5MB pictures, 80MB per page is HUUUGE). Thumbnailing would be great. I'm open for PRs ;)!
If anyone knows about a better alternative... I set this up when we got back from one vacation for my relatives to easily see the pictures (without social media).
Right now I have public (read-only) and private buckets only, and I'm the only who writes into any of them.
Public buckets contain files I didn't even create myself and that friends might find useful (Windows ISO, movies, VirtualBox VMs...). Privates have, well, private data, and can only be accessed using my admin account's credentials.
IIRC MinIO has access control through users, but I'm still very new to MinIO to the point where I discover new features every time I use it.
If I were to give someone else their own buckets I'd probably run a second instance to keep things separate, though. I'm even considering running another one myself to keep private buckets only accessible from my home network... (right now the entire instance is reachable from WAN, regardless of whether they are public or not).
I would be _very_ interested in a write up/explanation of this set up
Essentially, this setup achieves 5 features I wanted my DNS to have:
- Confidentiality: from my ISP; and from anyone listening to the air for plain-text DNS questions when I'm on public WiFi. Solution: DNS-over-TLS[1]
- Integrity: of the answers I get. Solution: DNS-over-TLS authenticates the server
- Privacy: from web trackers, ads, etc. Solution: domain name blacklist
- Speed: as in, fast resolution times. Solution: caching and cache prefetching[2]
- Observability: my previous DNS was Dnsmasq[3], AFAIK Dnsmasq doesn't log requests, only gives a couple stats[4], etc. Solution: a Prometheus endpoint
CoreDNS ticks all of the above, and a couple others I found interesting to have.
To set it up, I wrote my own (better) CoreDNS Docker image[7] to run on my Kubernetes cluster; mounted my Corefile[8] and my certificates as volumes, and exposed it via a Kubernetes Service.
The Corefile[8] essentially sets up CoreDNS to:
- Log all requests and errors
- Forward DNS questions to Cloudflare's DNS-over-TLS servers
- Cache questions for min(TTL, 24h), prefetching any domains requested more than 5 times over the last 10 minutes before they expire
- If a domain resolves to more than one address, it automatically round-robins between them to distribute load
- Serve Prometheus-style metrics on 9153/TCP, and provide readiness and liveness checks for Kubernetes
- Load the /etc/hosts.blacklist hosts file (which has just short of 1M domains resolved to 0.0.0.0), reloads it every hour, and does not provide reverse lookups for performance reasons
- Listens on 53/UDP for regular plain-text DNS questions (LAN only), and on 853/TCP for DNS-over-TLS questions, which I have NAT'd so that I can use it when I'm outside
The domain blacklist I generate nightly with a Kubernetes CronJob that runs a Bash script[9]. It essentially pulls and deduplicates the domains in the "safe to use" domain blacklists compiled by https://firebog.net/, as well as removing (whitelisting) a couple hosts at the end.
That's pretty much it. The only downside to this set up is that CoreDNS takes just short of 400MiB of memory (I guess it keeps the resolve table on memory, but 400MiB!?) and lately I'm seeing some OOM restarts by Kubernetes, as it surpasses the 500MiB hard memory limit I have on it. A possible solution might be to keep the resolve table on Redis, which might take up less memory space, but I'm still to try that out.
[1] Which I find MUCH superior to DNS-over-HTTPS. The latter is simply a L7 hack to speed up adoption, but the correct technical solution is DoT, and operating systems should already support it by now (AFAIK, the only OS that supports DoT natively is Android 9+).
[2] It was when I discovered CoreDNS' cache prefetching that I convinced myself to switch to CoreDNS.
[3] http://www.thekelleys.org.uk/dnsmasq/doc.html
[4] It gives you very few stats. I also had to write my own Prometheus expoter[5] because Google's[6] had a fatal flaw and no one answered to the issue. In fact, they closed the Issues tab on GitHub a couple months after my request, so fuck you, Google!
[5] https://github.com/ricardbejarano/dnsmasq_exporter
[6] https://github.com/google/dnsmasq_exporter (as you can see the Issues tab is no longer present)
[7] https://github.com/ricardbejarano/coredns, less bloat than the official image, runs as non-root user, auditable build pipeline, compiled from source during buil...
I learnt about CoreDNS because Kubernetes uses it for service discovery, and once I read about it's "chaining plugins" philosophy I wanted to try it out.
And it was so refreshing coming from Dnsmasq that I fell in love with it.
DoT is a protocol explicitly designed for it's purpose.
If brickhead sysadmins block DoT it's their problem, and if you have to work around that then it is, in fact, a hack (or a "workaround", doesn't matter).
It's not that DoT or DoH are superior to one another, it's that DoT is "DNS in TLS", and DoH is "DNS in HTTP in TLS", doesn't that raise a red flag for you?
The idea that end-users should give a shit about any of this "L7" "purpose built" "control plane" "layering violation" nonsense, and opt themselves into a version of DNS privacy that their network operators can turn off for them without end-user consent, is lunacy; bamboozlement.
I agree in that end-users shouldn't give a dime about DNS privacy, it should be private by default, but it is up to us to promote the correct protocol over the "hacky" one.
If DNS-over-HTTPS is superior, then why don't we shove everything down 443/TCP? Or better yet, why don't we get rid of TCP altogether and send everything over a port-less encrypted dynamically-reliable trasport protocol? Surely middleman couldn't distinguish between traffic.
Ports are there for a reason. The fact that they are used with anti-end-user intent doesn't make them (or any protocol that runs on them) inherently bad. Yet one thing that makes a protocol better than another one, given set of requirements, is efficiency.
By the way, if I were to switch my DoT server from 853/TCP to 443/TCP, the port wouldn't be a problem anymore. Per your standards, now DoT would be better than DoH, wouldn't it? Same results, smaller payloads.
I gave you an apples to apples protocol comparison. If you tell me there's a single bit that lets you distinguish between HTTPS traffic and DoT traffic running both on 443/TCP, then I'll buy your "kill switch" argument.
And even if you do, nothing keeps me from saying farewell to my ISP as soon as they press that switch.
I remember comparing low power homeservers, consumer NAS and a refurb Thinkpad and the latter won when considering the price/performance and idle power consumption (<5W). You also get a built screen & keyboard for debugging and a efficient DC-UPS if you're brave enough to leave the batteries in. That's of course assuming you don't need multiple terabytes of storage or run programs that load the CPU 24/7, which I don't. These days a rPi 4 would probably suffice for my needs but I still think the refurb thinkpad is a smart idea.
I do leave the batteries in. Is it dangerous? I read some time ago that it is not dangerous, but the capacity of the battery drops significantly, I don't care about capacity, and safe shutdowns are important to me.
In the past I used an HP DL380 Gen. 7 (which I still own, and wouldn't mind selling as I don't use it), but I had to find a solution for the noise. And power consumption was at around 18EUR for my EUR/kWh.
Cramming down what ran on 12 cores and 48GiB of RAM on a 2-core, 4GiB (I only upgraded the memory 2 months ago) machine was a real challenge.
The ThinkPad cost me 90EUR (IBM refurbished), we bought two of them, the other one burnt. The recent upgrades (8GiB kit + Samsung Evo 1TB) cost me around 150EUR. Overall a really nice value both in compute per EUR spent and in compute per Wh spent. Really happy with it, I just feel it is not very reliable as it is old.
Should be fine if they're not swollen / getting very hot
It's not necessarily dangerous but lithium batteries have a chance to fail and in very rare cases even explode, making them a potential fire hazard. I'm not an expert, maybe someone else can expand on this. If I were to run an old laptop of unknown provenance with a LiIon battery 24/7 completely unattended I'd at least want to make sure that it is on a non-flammable surface without any flammable items nearby.
>In the past I used an HP DL380 Gen. 7 (which I still own, and wouldn't mind selling as I don't use it), but I had to find a solution for the noise. And power consumption was at around 18EUR for my EUR/kWh.
Yes, I am surprised how many people leave power consumption out of the equation. These days you can rent a decent VPS for the power cost of an old refurb server alone.
Well, I'm removing the battery and the pseudo-UPS logic right now. The battery looks fine, but I'm not taking any risks, since it's on top of the DL380 but under a wooden TV stand.
Thanks for the heads up! You might have prevented a fire.
Here’s my home lab: https://imgur.com/a/aOAmGq8
I don’t self host anything of value. It’s not cost effective and network performance isn’t the best. Google handles my mail. GitHub can’t be beat. I use Trello and Notion for tracking knowledge and work, whether personal or professional. Anything else is on AWS. I do have a VPN though so I can access all of this when I’m not home.
The NAS is for backing up critical data. R720 was bought to experiment with Amazon Firecracker. It’s usually off at this point. Was running ESXI, now running Windows Server evaluation.
The desktop on the left is the new toy. I’m learning AD and immersing myself 100% in the Microsoft stack. Currently getting an idiomatic hybrid local/azure/o365 setup going. The worst part about planning a MS deployment is having to account for software licensing that is done on a per-cpu-core basis.
People who use bittorrent legally do exist, or at least there's one of us.
I also have piracy.
How would _you_ suggest I handle the 2TB of public domain media I have, then?
The status quo is radically anti-consumer, IMO, as radical as abolition of all copyright would be.
Of all the ways to try to promote creativity in the 21st century, making information distribution illegal by default and then using force of law to restrict said distribution unless authorized is pretty wack.
It makes sense when you consider that information is generated in the first place for an incentive, and that incentive is only possible when copyright guards it. People are more than free to create public information if they choose to do so (and they do), but some people generate valuable information mostly for the purpose of profiting from it and the copyright framework tries to ensure that it will be worth their time when they attempt to create such information. Would you rather they didn't have the option which would result in the effort not being expended to generate such information? With copyright, you at least have the option to obtain it if you deem the price tag (set by the creator) fits the value you'll get from it.
There is no central authority that copyrights information that people generate. You make it sound like there is some evil force in the world that prevents people from creating freely accessible information. There isn't. You're free to create freely accessible information. There are creators that choose to limit access to the information that they generate and I don't understand how someone can argue that it is unfair that they have an option to do so if they choose.
Presumably so because consumers stealing the final products was already prohibited / a crime. The final product was generally physical, and consumers would have to physically break into stores to get their copy of whatever was produced and it was already illegal. And even if the consumers obtained their copy by legitimate means, them sharing with other would mean they would lose their own copy. For consumable-ish items (things you only need to experience once to get the value out of the product) this is still a problem of course but there is no easy way of preventing it - but the idea is still there and the limits can be enforced. With digital information, the barrier for entry for such theft is greatly reduced. You don't lose your copy when you share, and stealing is a lot easier too. Doesn't mean it is right or it is in line with the spirit of what we thought ownership meant back in the day.
>Copyright as it is goes against the very purpose of creation preventing any new works from ever being created.
Again, I don't get this. EVERYONE has the OPTION to create works for public domain. Why is this not enough for you? Everything you want is already there. It's just that there is another option for others that don't want to create works for public domain. Why does that bother you?
My guess is that if you were entirely happy with what people create without a motive for profit, you wouldn't care that other people had a copyright option. But you are not happy with what that economic model (free, copyleft etc.) produces by itself. You are aware that that economic model doesn't work. You want free access to information that people with economic incentives create with a price tag attached to it, because you know information generated with profit in mind tends to be more valuable.
https://www.smithsonianmag.com/arts-culture/first-time-20-ye...
> Would you rather they didn't have the option which would result in the effort not being expended to generate such information?
Yes, I would argue absolutely that valuable information would still be made because those that would benefit just from the information existing - not from the potential sale of said information - would still fund its creation. Someone that wants a painting will still pay for it if or if not they can sell the finished work. The think tank researching a cure for cancer will still have ample funding sources from those who think not having cancer would be beneficial regardless of those sponsors ability to profit off said cure.
> I don't understand how someone can argue that it is unfair that they have an option to do so if they choose.
Its largely a problem because its both default and implied. Its in the same class of problem as if the government tried to restrict air - you had to pay to breathe and are charged per-month. Despite the air being "free" and "everywhere". Its a tough analogy to write though, because there is no true analog to the modern miracle of information propagation being infinite and endless - we truly have nothing else worth so little as a copy of a number to compare it to.
But fundamentally its having your cake and eating it too - if you want to monetize your creations, you make them for free (at expense to yourself) to try to monetize something that has no value (copies of it). Its so abjectly opposed to reality and true scarcity that subconsciously drive people to feel no serious shame in piracy despite them "stealing theoretical profits from the rightsholder". But thats really all you are taking. In another light, a random stranger is offering you something for free and without recompense that they have, just because its so cheap to store, transmit, and replicate. That is magical. We take this modern miracle of technology and bind it in chains to try to perpetuate a model of profit that doesn't make any actual sense in actual reality given the scarce inputs (creative capability, motivation, and efforts) and infinite outputs (information) involved.
You can though. Precisely because there are countries (past and present) that have / had no legal framework for such incentives, and there are others where the framework was there but the law is not enforced. And we can observe how it is working for them, compare and contrast. I live in one of them (lived here all my life) but work for countries that provide such a protection (precisely because my intellectual property would not be respected here, so my own country my homeland does not get to benefit from my work) so it is easy for me to look at both sides of the coin - though it is not strictly necessary. It doesn't take hands on experience to observe that the most productive and innovative countries occupying planet earth are those that have strong intellectual property rights.
There is a lot of low hanging fruit where I live that would double the GDP of the country in a few years but no one is doing it, because they are either capital intensive, or time intensive (or both) but without any protections there to make it worth your while it doesn't make sense to attempt those - for anyone. It makes more sense to pitch any innovative ideas to countries that will protect you so that you get a return proportional to the value you generate for the rest of the society. If I have an idea that has the potential to shave 1 hour off of millions of people's work everyday, that is enormous value generated for everyone and I should be rewarded proportionally. Not to mention the risk I'd have to take to attempt that, by attempting that I'm doing this instead of doing something else with my only life so of course the incentive HAS TO be there.
>The think tank researching a cure for cancer will still have ample funding sources from those who think not having cancer would be beneficial regardless of those sponsors ability to profit off said cure.
I think this is far too naive way of looking at it. You are taking risk entirely out of the equation. If I'm attempting to do something that is of value to other people, merely by attempting it, I am taking a RISK. My time is my most valuable asset, I only live once, and I'm willing to invest my time and capital into this endeavor instead of doing something else with them. The resources of human beings are not unlimited, so they need to have a heuristic / algorithm to ration their resources. Their survival / wellbeing is dependent on this. So something becomes a viable risk only if there is a possible return that makes sense. Like you wouldn't play coin flip for 1.1x or nothing right? It must at the very least be 2x or nothing to break even.
So we think we'd like to do good for no possibility of personal return, but behavioral economics show that humans do not operate that way (even though they'd like to think that they would) because each and every human being have their own lives, responsibilities, families, wants, wishes and only limited resources. To make ANYTHING the returns must be congruent with your heuristic about how you'd like to divide your limited resources.
You are still approaching it from the "make it for free, charge for the result" model incompatible with reality. If there is something worth doing that will radically improve society you should be soliciting the funding first before undertaking the labor.
Yes, risk is involved. You take risks when you pay someone to build a house for you. You take risks when you buy Sushi at the store that may or not still be good. We have, as a society, very effectively structured and produced buyer protections for services rendered ranging from guarantees to total free for all. You would, like with every other transaction, budget and account for risks and pay to mitigate them if warranted.
> It makes more sense to pitch any innovative ideas to countries that will protect you
Its more like foreign nations with IP laws are offering you a magic money machine that nations without don't, and the pile of gold is a tempting proposition over attempting alternative funding models.
> So something becomes a viable risk only if there is a possible return that makes sense
I don't think we are in any disagreement here. I'm never arguing that you abolish IP and expect all further scientific advancement, art, programming, etc to be done by people who cannot seek compensation for making it. I'm arguing that the US IP regime smothers any potential alternative with how exploitative having the government constrain information by law is.
Your story runs a similar thread seen in tax evasion and money laundering - the rich will gravitate their wealth towards wherever they can keep the most of it. That is where Swiss Bank Accounts and Latin American cartels get their power. Likewise businesses want lower taxes and thus gravitate towards the countries that offer the lowest tax rates - even if those lower tax rates have abject demonstrable harm to the citizenry through reduced social programs, etc. The result is that there is a global race to the bottom economically - to appeal maximally to wealth to attract it, or see your nation rot as it constantly flees your borders for greener pastures. IP is a massive pile of money to be had, hence nations without it see creators flee to nations with it, but might does not make right - just because the existence of IP represents untenable profit by any other means does not justify its existence, especially when it is so contrarian to baseline reality. Its a perversion of normality meant to attract investment the same way having low or no wealth and income taxes or no corporate tax attracts the wealthy and businesses.
> So we think we'd like to do good for no possibility of personal return
The personal return on funding the cure for cancer is having the cure for cancer exist.
> To make ANYTHING the returns must be congruent with your heuristic about how you'd like to divide your limited resources.
And macroeconomically we are all constituted of limited resources - limiting information compels most to partition their scarce resources towards affording the IP regime, not for any physical necessity, and this reduces the buying power of everyone involved. IP falls under the same purview of economic cancers as advertising, health insurance, and the military - bureaucracy and a race to the bottom that has no abject benefit but siphons productivity away to rent seekers and middle men.
I specifically didn't mention music because it's easy to get it DRM-free. Pretty much every online music store is DRM-free.
> Bums me out when I see people putting so many resources into running/building elaborate piracy machines.
These two comments are rather at odds to me.
That said, IME generally the type of person who's big into self hosting isn't a Microsoft guy. I work with MS stuff at work at the moment. The entire thing is set up for Enterprise and Regulations. It's hugely overcomplicated for that specific goal only.
At home I don't care about Regulations(tm). The only reason I can see for someone to bother with it is if they want to train out of hours for a job at an MS shop.
It seems to hold rack-mounted gear quite well.
Looks like the IKEA IVAR storage system. https://www.ikea.com/kr/en/catalog/categories/departments/li...
I've mostly fallen into it at my job because the alternative to me pushing dev services to SAAS offerings and maintaining the glue myself is a pile of poorly-maintained IT-provided Server 2008 R2 boxes.
I notice I was a lot more keen on hosting a bunch of crap myself before I knew how to do it "right", and before devops, orchestration ("you mean running scripts in remote shells?"), cloud, or containers or any of that were things. And yet it all worked just fine back then—time spent fixing problems from my naïve "apt-get install" or "emerge" set-up process wasn't actually that bad, compared with the up-front cost of doing it all "right" these days. A couple lightly-customized "pet" servers were fine, in practice. Hm.
So then look at home projects and I wonder if I know enough to self host things, or host them on GCP in a manner that won't just invite getting hacked, running up a ridiculous bill, or leaking my private sensitive data out.
Any guidance to offer?
2) A lot of what people do is chasing nines that you don't need (and a lot of the time they don't either, but "best practices" don't you know, and no-one wants to have not been following best practices, even if doing so was more expense and complexity than it was worth for the company & project, right?) so just forget about failover load balancers and rolling deploys and clustered databases and crap like that. All of that stuff can be ignored if you just accept that you may have trouble achieving more than three nines.
3) If it's just for you, consider forgetting any active monitoring too. That can really kill your nines of reliability, but if it's mostly just you using it, that may be fine, and you won't get alerts at 3:00AM because some router somewhere got misconfigured and your site was unreachable for two minutes for reasons beyond your control. Otherwise use the simplest thing that'll work. You can get your servers to email you resource warnings pretty easily. A ping test that messages you when it can't reach your service for the last X of Y minutes (do not make it send immediately the first time it fails, the public Internet is too unreliable for that to be a good idea) is probably the fanciest thing you need. Maybe you can find some free tier of a monitoring service to do that for you and forget about it, even.
4) If you can mostly restrict yourself to official packages from a major distro, and maybe a few static binaries, it's really easy to just write a bash script that builds your server from scratch with very high reliability. Maybe use docker if you're already comfortable with it but otherwise, frankly, avoid if you can and just use an official distro packages instead, as it'll complicate things a lot (now you have a virtual network to route to/from/among, probably need a reverse proxy, you may have a harder time tracking down logs, and so on). Test it locally in Vagrant or just plain ol' Virtual Box or whatever, then let it loose on a fresh VPS. If you change anything on the VPS, put it in the script and make sure it still works. If you're feeling very fancy learn Ansible, but you'll probably be fine without it.
5) For security, use an SSH key, not a password, and change your SSH port to something non-default (put that in your setup script) just to cut down on failed login noise, if you feel like it. You could add fail2ban but if you've changed the port and are using a key it's probably overkill.
6) Forget centralized logging or any of that crap. If you have a single digit count of VPSen then your logging's already centralized enough. If one becomes unreachable and can't be booted again and you can't find any way at all to read its disk, and that happens more than once, consider forwarding logs from just that one to another that's more reliable if you wanna troubleshoot it. You can do this with basic logging packages available on any Linux distro worth mentioning, no need to involve any SaaS crap.
7) Backups. The one ops-type thing you actually have to to do if your data's not throwaway junk is backups. Backups and strictly-used build-the-server-from-scratch + restore-from-backup scripts are kinda sorta all most places actually need, despite all the k8s and docker chatter and such.
8) Cloudflare exists, if you have any public-facing web services.
[EDIT] mind none of this will help you get a job anymore since everyone wan...
* Unbound for dns-over-tls and single point of config hostnames for my home network
* Syncthing for file sync
* offlineimap to backup my email accounts
* Samba for a home media library
* cron jobs to backup my shares
* Unifi controller
On my todo list:
* Scheduled offsite backup (borg + rsync.net being the top contender currently)
* Something a bit more dedicated to media streaming than smb. some clients like vlc handle it fine, others do not.
* Pull logs for my various websites locally
On the front end I have two 1Gbit circuits (AT&T and Google) going into an OPNSense instance doing load-balancing and IPS running on a Dell R320 with a 12-thread Xeon and 24GB of RAM
Services are hosted on a Dell R520 with 48GB RAM and two 12-thread Xeons running Ubuntu and an up-to-date ZFS on Linux build.
Media storage handled by two Dell PowerVault 1200 SAS arrays.
Back-end is handled by a Cisco 5548UP and my whole apartment is plumbed for 10Gbit.
Holy hell. How did that come about?
Overleaf: https://sdan.xyz/latex
A URL Shortener: https://sdan.xyz
All my websites (https://sdan.xyz/drf, https://sdan.xyz/surya, etc.)
My blog(s) (https://sdan.xyz/blog, https://sdan.xyz/essays)
Commento commenting server (I don't like disqus)
Monitoring (https://sdan.xyz/monitoring, etc.)
Analytics (using Fathom Analytics) and some more stuff!
https://github.com/dantuluri/sd2/blob/master/docker-compose....
You'll need Mongo and Redis (last I remember) as well (which I believe are the two images that follow the sharelatex image.
I see a lot of people putting their home stuff behind CloudFlare, but when I reviewed their free tier, I didn’t actually see any security benefit to outweigh the privacy loss, and I didn’t see that covered on your blog post.
1. This is hosted on GCP. Actually was thinking of using Cloudflare Argo once my GCP credits expire so that I can truly self host all this (although all I have is an old machine).
2. For me, Cloudflare makes my websites load faster on pages. Security wise, I have pretty much everything enabled... like always on HTTPS, etc. and I some strict restrictions on SSHing into my instance (also note that none of my ip addresses are exposed thanks to Cloudflare), so really I'm not sure what security risk there may be.
3. How am I losing privacy loss? Just curious, not really understanding what you're saying there.
Browser <> CF, CF<> source server. Two distinct TCP sessions. Both potentially encrypted, but there’s no E2E encryption anymore.
Can you elaborate on this? Maybe I misunderstand you, but is there a good way to get HTTPS from your home?
1. Been using it for 3+ years. Whenever I'm making a site, there's nothing better than easily making some DNS records and making sure they're all always-on HTTPS.
2. A little bit of the first part: It's a hassle to setup my own certs, etc.. I feel that Cloudflare "protecting" my IP from DDOS and other attacks is far better than anything that I can setup easily (at least from my experience, I think they know what they're doing)
3. Maybe in the future when I have some time and money I'll do everything on my own and ensure I have E2E encryption. At the moment, anything I'm running isn't mission-critical and isn't used by hundreds of people; I'm not making a SaaS startup. I understand your concern, but the ease of use of Cloudflare is something I value.
4. Analytics. I've come not to trust Google Analytics at all. I'm not sure what they're doing, but most if not 100% of tech-savy people have adblock, which blocks GA. My VPN from AlgoVPN blocks GA and anything related to GA, FB, Twitter, etc. So I'm not really sure how much I can trust GA's analytics opposed to Cloudflare giving me the exact numbers on how many people requested or visited my site. (I'm going to make my own analytics soon since Fathom has turned to profit only and not open source).
> 3. How am I losing privacy loss? Just curious, not really understanding what you're saying there.
I understand the benefit of CF, and it’s for each person to decide for themselves what they consider acceptable or not.
> Cloudflare giving me the exact numbers on how many people requested or visited my site. (I'm going to make my own analytics soon since Fathom has turned to profit only and not open source).
We could do this in the 90s with our web server logs. It didn’t involve third parties or paid tools or centralising logging and sacrificing privacy. Tooling for simple stats has existed for literally more than two decades.
I'd suggest that Argo is a waste of money if you have control of your router, you don't need to secure unencrypted HTTP traffic, and your ISP isn't port-blocking. Block all traffic except from CF's IPs, configure Authenticated Origin Pulls, and use SSL for your CF<->Origin traffic (your own cert or CF's).
If you don't meet all of those conditions, a cheap VPS as a VPN server is probably a better value (plus you get a VPS to do other stuff with).
The main thing is being able to hide your origin IP address. That turns many types of DDoS attacks into CloudFlare's problem, not yours, and it doesn't matter that you're on the free tier[0]. If you firewall to only allow traffic from CF[1], then you can make your services invisible to IP-based port scans / Shodan.
CloudFlare isn't a magic-bullet for security, but, used correctly, they greatly reduce the attack surface.
Whether any of that is worth the privacy / security risk of letting CloudFlare MITM your traffic is up to you.
[0] https://news.ycombinator.com/item?id=21170847
[1] https://www.cloudflare.com/ips/
I wrote this to setup my web server, mail server and VPN server, and auto-generate all my VPN keys.
https://github.com/sumdog/bee2
I'm probably going to change how publicly accessible my monitoring view is soon, but for now, it seems pretty cool for everyone to see.
Would love to get a link to a screenshot of your system's resource monitoring. The description of each panel & eache metric was quite useful!
But at the same time, I understand the security risks and if I have to I can just stop netdata's container and add some more security on it before turning it on again (I'm not running some SaaS startup, so security isn't a huge concern and I don't think you can do anything with my netdata that can affect or show anything else that can make me prone to attack)
Of course, it would simplify privilege escalation if someone successfully attack netdata service. If you want public dashboard, streaming is supposed to be quite safe (no way to send instruction to streaming instance of netdata).
Home automation/security system + 'Alexa': completely home grown using python + android + arduino + rpi + esp32
It all started with hosting subsonic
Have been meaning to move more to colo, especially my Wordpress install and some Wordpress.com-hosted sites, but inertia.
[0] https://support.cloudflare.com/hc/en-us/articles/204899617-A...
[1] https://www.cloudflare.com/ips/
I use one for the sites below. It is written in Java/Kotlin, but barely works anywhere except Windows.
https://egov.kz/cms/en
https://cabinet.salyk.kz/
...
Home: Two VMware hosts on Hyve Zeus (Supermicro, 2xE5 64GB), one on an HP Microserver Gen8 (E3-1240v2 16GB). PiHole bare metal on a recycled Datto Alto w/ SSD (some old AMD APU, boots faster than a Pi and like 4w). Cloud Key G2 Plus for UniFi / Protect.
VMware because it's what I'm used to. Hyper-V because it's not. Used to have some stuff on KVM but :shrug:
I've always been unable to pull this off completely as I always want a way to SSH into my home network - but maybe there is a better way I can pull off this sort of 'break glass' functionality.
Guacamole (sorta) gives me that. If CloudFlare or nginx or Guacamole have problems then I'm hosed... but I work from home so remote access isn't a huge concern.
And I've got nothing terribly "household critical" at home, just the PiHole needs to be running to keep everyone happy. I do wish that PiHole had an HA solution. I've been tempted to set up a pfSense / pfBlockerNG HA pair but that's a lot of overhead just for DNS.
You could run 2 Pi’s or a Pi and a container in another always on machine for example. Then just point your router‘s primary to the Pi and secondary to the other instance.
IMO, using a Tor hidden service is a (damn near) perfect solution for this.
have you made it work? my Tor career ended in college after running an exit node - no visits from the FBI, just got auto-klined from every IRC server since I was on the list of proxies.
[0] https://github.com/huan/docker-simple-mail-forwarder