The beauty of kube is the consistency and portability of charts and deploys between clusters (when you aren’t accounting for cloud specific stuff like IAM, LB, etc). So I don’t find this to be as crazy as it initially sounds.
A couple of years ago, I was dabbling in the cloud at AWS, and I initially decided to run a MediaWiki server with its associated stack, such as MariaDB and Apache or something, on Debian. And I found a good Bitnami AMI to do it with.
But what I really wanted to do was to spin up Kubernetes and learn about microservices in some way. The techie guys I polled said I wouldn't need k8s at all for my use case and I should scrap trying to do k8s because it wouldn't do me any good. I'm still sort of miffed at that; I would've rather learned some Kubernetes in the first place rather than just become a petty sysadmin again, so a replacement project that actually did make use of k8s would've been helpful.
Anyway I chose to double down on my chosen career in the education industry, so I'm letting my sysadmin/DevOps skills lie fallow until retirement.
Honestly, getting started with k8s is easy. Get started with minikube on local desktop. Once you get the feel for it, try configuring a multi-node cluster using k3s on DigitalOcean or Hetzner and eventually using managed AKS/EKS/GKE if you feel the need.
I was surprised when I setup k3d the first time, I had a multi-node cluster on my (4 year old) desktop and was able to deploy a multi-node service I wanted to test out without any problems. I would highly recommend k3d if you don't want to pay for cloud servers and don't actually need either high-availability or the CPU power of 3 servers. Also, your desktop is probably more powerful than any base-tier server.
What do you use for load balancing and ingress? Do you rely on service discovery in this setup? I always get stuck figuring out how to get traffic into the cluster properly while allowing for graceful fault tolerance.
I’ve had success with MetalLB to dynamically announce a single IP for the ingress controller but that feels a bit overkill for a simple lab setup.
For my clusters, they're all single-node and therefore not fault-tolerant clusters, so I don't worry about graceful fault tolerance. I just use nginx-ingress which opens a port on the node (via iptables, so don't expect `lsof` to help!). Then my router port-forwards so my home IP:80 eventually hits my NUC
I'm interested in Kubernetes the same way I'm interested in Erlang. I find distributed systems so much fun to reason about, and trying to understand certain implementation details and design choices shed a light on the reasoning of very smart people on very hard problems.
Now, would I want to deal with those technologies all day? Not really. I also like diesel electric locomotives, but I don't wanna run a fleet of them or run through their maintenance sheets.
That's the advantage of a homelab environment - the whole monolith/microservice complexity & technical debt thing just doesn't matter in that context. Anything interesting is fair game.
One more more "Why" motivation to add: Portability. I can stick it on my desktop (minikube), my homelab cluster, or a DIY cloud cluster or CP. Feels a bit like docker in that sense - common lowest denominator to build against.
> If anyone knows of a CLI that would give me that level of information, please leave a comment.
k9s tool is much better. Takes a bit of getting used to since it has the whole ssh old school vibe but its 10x more useful that the official dash.
>prometheus/alertmanager: I don't have this setup
I've found Promtail daemonset connected to Loki/Grafana to be good. Especially if you stick ever deployment into it's own namespace, since you can then use that as label in Grafana/Loki. Had a couple issues that I could only track down once I had external logging.
I know about k9s, but it doesn't have a "Workloads" tab that shows both failing pods and their source (deployment, daemonset, etc).
> I've found Promtail daemonset connected to Loki/Grafana to be good.
I'll look into this, thanks!
> Another neat trick is using Kompose to translate docker-compose to yaml. Helm works too, but I find them a little opaque.
I didn't mention how I wrote/got all the services' yaml, but it was a combination of kompose (and one in instance an online tool that converted a 'docker run' command I had hand-crafted a while ago to docker-compose, to chain into Kompose), chatgpt for the basic ones, and helmCharts via kustomize.
9 comments
[ 3.2 ms ] story [ 17.7 ms ] threadBut what I really wanted to do was to spin up Kubernetes and learn about microservices in some way. The techie guys I polled said I wouldn't need k8s at all for my use case and I should scrap trying to do k8s because it wouldn't do me any good. I'm still sort of miffed at that; I would've rather learned some Kubernetes in the first place rather than just become a petty sysadmin again, so a replacement project that actually did make use of k8s would've been helpful.
Anyway I chose to double down on my chosen career in the education industry, so I'm letting my sysadmin/DevOps skills lie fallow until retirement.
I’ve had success with MetalLB to dynamically announce a single IP for the ingress controller but that feels a bit overkill for a simple lab setup.
For my clusters, they're all single-node and therefore not fault-tolerant clusters, so I don't worry about graceful fault tolerance. I just use nginx-ingress which opens a port on the node (via iptables, so don't expect `lsof` to help!). Then my router port-forwards so my home IP:80 eventually hits my NUC
Now, would I want to deal with those technologies all day? Not really. I also like diesel electric locomotives, but I don't wanna run a fleet of them or run through their maintenance sheets.
That's the advantage of a homelab environment - the whole monolith/microservice complexity & technical debt thing just doesn't matter in that context. Anything interesting is fair game.
One more more "Why" motivation to add: Portability. I can stick it on my desktop (minikube), my homelab cluster, or a DIY cloud cluster or CP. Feels a bit like docker in that sense - common lowest denominator to build against.
> If anyone knows of a CLI that would give me that level of information, please leave a comment.
k9s tool is much better. Takes a bit of getting used to since it has the whole ssh old school vibe but its 10x more useful that the official dash.
>prometheus/alertmanager: I don't have this setup
I've found Promtail daemonset connected to Loki/Grafana to be good. Especially if you stick ever deployment into it's own namespace, since you can then use that as label in Grafana/Loki. Had a couple issues that I could only track down once I had external logging.
https://grafana.com/docs/loki/latest/clients/promtail/instal...
Another neat trick is using Kompose to translate docker-compose to yaml. Helm works too, but I find them a little opaque.
> k9s tool is much better
I know about k9s, but it doesn't have a "Workloads" tab that shows both failing pods and their source (deployment, daemonset, etc).
> I've found Promtail daemonset connected to Loki/Grafana to be good.
I'll look into this, thanks!
> Another neat trick is using Kompose to translate docker-compose to yaml. Helm works too, but I find them a little opaque.
I didn't mention how I wrote/got all the services' yaml, but it was a combination of kompose (and one in instance an online tool that converted a 'docker run' command I had hand-crafted a while ago to docker-compose, to chain into Kompose), chatgpt for the basic ones, and helmCharts via kustomize.