Show HN: GitOps Template for Kubernetes (github.com)
Nowadays, it is best practice to use Git as a revision control system for your Kubernetes configurations. Update automation workflows like Renovate or Dependabot can create pull requests for new versions of Docker images and Helm charts, but ensuring these new package versions work is still a manual task. By using the central (or a private) Glasskube repository (https://github.com/glasskube/packages) together with our Renovate integration (https://docs.renovatebot.com/modules/manager/glasskube/), you can ensure that new package versions will run through our Minikube-based CI workflows before they get published—similar to how the Homebrew core tap works. We’ve just introduced readiness checks for manifest-based deployments and utilize the flux-helm-controller to wait for a Helm release to succeed.
Dependencies are resolved by our package controller. These dependencies can either be cluster-scoped (installed in the recommended namespace, e.g., operators wird CRDs) or namespace-scoped components of a package (e.g., a database or Redis cache). In such cases, we will prefix resources with the dependent package name to ensure multiple packages can use the same dependencies without naming conflicts (we use Kustomize on a virtual filesystem for this).
Glasskube packages can currently be Helm charts (from an OCI or Helm repository) or manifests, which are mostly built using Kustomize’s overlay approach.
Since neither the overlay approach (using Kustomize) nor Helm’s limited templating functionality will help us and other Kubernetes users scale to more complex packages, we are considering creating a more programmatic approach to package creation, similar to Timoni. Currently, KCL is our frontrunner (https://github.com/glasskube/glasskube/discussions/1018), as it already integrates well with the Kubernetes ecosystem.
We would appreciate if you give our GitOps template a try. It also works work existing Kubernetes clusters if just want to use GitOps for some applications. Just make sure that the argocd and glasskube-system namespaces are not yet in use. See: https://github.com/glasskube/gitops-template/
36 comments
[ 3.4 ms ] story [ 93.0 ms ] threadI assume this statement is for running this? glasskube bootstrap git --url <your-repo> --username <org-or-username> --token <your-token>
I think I'd like to understand what the argo cd / git ops template is and how its different than argocd autopilot. Maybe some pictures of how argo is deploying apps. Etc.
The `glasskube install` command does a bunch of stuff that ends up as resources in your Kubernetes cluster, that are then interpreted by the Glasskube operator.
The "Gitops template" make use of ArgoCD and Git to do what `glasskube install` would have done.
I am not super thrilled about critical applications like Argo getting a plethora of plugins, otherwise we end up looping back to Jenkins and plugin hell
On-topic: Glasskube isn't really an ArgoCD plugin as it can work standalone, but in 2024, can you really propose a package manager for k8s without having some integration with ArgoCD and GitOps in general?
If you want to migrate, having interoperability between the tools can make the process smoother. And if you don't want to migrate and still benefit from a centralized, curated, audited repository of packages for Kubernetes so that your "Powered by ArgoCD" GitOps are easier to manage, that's what the GitOps template propose.
In Debian, you can just `apt install <that big thing i don't want to write a deploy script for>`. Imagine doing that with the usual big operators you want in your cluster (cert-manager, a hashicorp vault operator, istio or nginx ingress controller or envoy or ...)
I certainly prefer FluxCD, myself.
FluxCD is fully batteries-included, but the UI (3rd party!) leaves a lot to be desired, and turns off a lot of developers, and as a result makes it difficult to get team buy-in
ArgoCD is missing some critical systems, like how to tell when the underlying image needs updating. There are a number of ways to handle this but it's either a kludge, a plugin, or both. However, the UI is fantastic and very easy to pickup and understand, team buy-in is usually close to 100%
I generally recommend starting with argo, and once the team/project/s have matured, migrate to FluxCD. Eventually you want to lock everyone out of the CD system, but initially people are skeptical and want to understand/watch everything work, especially during debug.
Our bootstrap git sub command is similar to argo cd autopliot. I give it a try right now to be able to better state differences and follow up on this question.
Our template already includes update automations with the renovate integration and easier integration of third party applications.
Tldr.: If you are already using open shift, make use of the operator hub, else glasskube is the more lightweight and simpler solution with similar concepts.
kubectl create -f https://operatorhub.io/install/flink-kubernetes-operator.yam...
[1] https://operatorhub.io/how-to-install-an-operator
We have abstracted all packages as custom resources and have a controller that reconciles these resources to (1) enable drift detection. Additionally, we use admission controllers (2) to validate dependencies and package configurations before they are applied to the cluster, while also working with custom resources to store and update the status of installed packages.
Can we benefit your project in a more limited but agentless way? Limiting the types and CRDs we allow in k8s makes operations better, especially with the aggressive upgrade cycle that k8s already imposes.
I don't think that idea of gitops is broken if the glasskube package controller and all custom resources are versioned you will always lead to a reproducible result.
> Can we benefit your project in a more limited but agentless way?
We are building a central package repository with a lot of ci / cd testing infrastrucutre to increase quality of kubernetes packages in general: https://github.com/glasskube/packages
a.) in Kubernetes setups that operate the same software stack, with the ongoing updates and regular releases.
b.) in Kubernetes setups that frequently install new software/diverse software
If your setup become more complex and changes often (b) a I would recommend breaking it up into smaller pieces.
For both scenarios it makes sense to use git to keep track of revision and previous states of your Kubernetes cluster and incorporate review processes with pull requests.
Sure, you could make your own .deb, your own repository, and manage dependencies yourself. But do you really want to?
See, there's a bunch of stuff that I deploy using terraform (VPC, DNS, EKS) and a bunch of stuff that I deploy with FluxCD. But in between, there's some awkward stuff like the monitoring stack that requires cloudy things and Kube things, and they're tightly coupled.
Right now I end up going with terraform and the awful helm provider. Many of these helm charts have sub-charts, nested values etc, but thankfully the monitoring stack doesn't change that much. It's still not ideal but it works as a one-shot deployment that sets up buckets, policies, IAM roles for service accounts, more policies for the roles, and finally the helm charts with values from the outputs of those clouds resources.
I've tamed terraform, mostly. Individual providers can be awful though.
https://github.com/hashicorp/terraform-provider-helm/issues/...
Any time I touch the monitoring stack, which has several helm releases with large values blocks (kube-prometheus-stack, promtail, Loki, Mimir), it's absolutely nightmarish. The plan can be hundreds of lines that have to be diffed manually.
I recommend giving Glasskube a try with Minikube locally and join our Discord to interact with the community.
I want a single code project that describes and deploys my monitoring stack. With terraform and the helm provider I can create cloud resources with terraform and deploy the kube resources using the terraform helm provider, using values that come from the outputs of the terraform cloud resources, in a single operation.
I don't think glasskube can replace helm in this instance. Wouldn't I have to split my monitoring stack into cloud ops and Kube ops, and manually paste outputs from terraform into glasskube configs?
How does it compare with OpenShifts operator hub?
is glasskube a reboot of jenkins-x ?