Ingress as defined by Kubernetes is really restricted if you need to do rewriting, redirecting and basically all the stuff we've been doing in pre-Kubernetes times. Nginx Ingress Controller worked around that by supporting a ton of annotations which basically were ingested into nginx.conf, to the point that any time you had a need everyone just assumed you were using nginx-ingress and recommended an annotation or two.
In a way, it was a necessity, since Ingress was all you'd get and without stuff like rewriting, doing gradual Kubernetes migrations would have been much more difficult to impossible. For that reason, every ingress controller tried to go a similar, but distinctly different way, with vastly incompatible elements, failing to gain traction. In a way I'm thankful they didn't try to reimplement nginx annotations (apart from one attempt I think), since we would have been stuck with those for foreseeable future.
Gateway API is the next-gen standardized thing to do ingress, pluggable and upgradable without being bound to a Kubernetes version. It delivers _some_ of the most requested features for Ingress, extending on the ingress concept quite a bit. While there is also quite a bit of mental overhead and concepts only really needed by a handful of people, just getting everyone to use one concept is a big big win for the community.
Ingress might not be deprecated, but in a way it was late to the party back in the day (OpenShift still has Route objects from that era because ingress was missing) and has somewhat overstayed its welcome. You can redefine Ingress in terms of Gateway API and this is probably what all the implementers will do.
Another triumph for open source: popular project probably used by many megacorps only propped up by the weekend charity of a couple unpaid suckers over the years.
How do you people even keep up with this? I'm going back to cybersecurity after trying DevOps for a year, it's not for me. I miss my sysadmin days, things were simple back then and worked. Maybe I'm just getting old and my cognitive abilities are declining. It seems to me that the current tech scene doesn't reward simple.
If your infrastructure can justify the complexity of Kubernetes, keeping up with Kubernetes native software is extremely easy comparing to anything else I have dealt with. I had some horror story managing nginx instances on 3 servers with ansible. To me that's much harder than working with ingress controllers in Kubernetes.
Replacing an ingress controller in Kubernetes is also a well documented practice, with minimum or even zero downtime if you want to.
Generally, if your engineering team can reasonably keep things simple, it's good. However, business needs to grow and infrastructure needs to scale out. Sometimes trying too hard to be simple is, in my experience, how things become unmanageably complex.
I find well-engineered complexity to be much more pleasant to work with.
Nginx annotations is not something we want to keep though and there is rarely such a thing as a seamless drop-in replacement. Every change needs to be intensively tested and nobody in their right mind would replace an ingress controller by another one in-place on any cluster size. The correct way will be to offer a new ingress class and nudge the cluster users to migrate to that one, one app at a time, which means you could as well migrate to something completely different, depending on size and effort. Traefik tries very hard to be an option in Kubernetes space, with some promising concepts, but supporting random annotations is not it.
This is terrible. Of all things k8s, ingress was the part I just did not want to have to mess with. It just worked and was stable, this gateway is completely unnecessary. And it seems to me that nginx retiring is just because people were pushing for the gateway so much that they threw in the towel. Infra is not react, people need to leave it alone.
I don’t think this is the https://xkcd.com/2347/ of the ops world? People will usually use the ingress controller of their cloud provider. I’ve been using the tailscale ingresses for tailscale funnel. But the transition from ingress to gateway api is seeming to take forever so I’m just running a caddy pod with a static config until the dust settles.
When I was choosing ingress controller few years ago, I think it was the most popular ingress controller by far, according to various polls. As I didn't have any specific requirements, I chose it and it worked for me. Over years I've used few proprietary annotations, so migrating away going to be a bit of pain. Not awesome news.
We have been building an ingress Nginx compatibility layer in Traefik that supports the most used ingress Nginx annotations. You should definitely give it a try as it makes Traefik a drop-in replacement to ingress Nginx, without touching your existing ingress resources.
Your feedback will be super useful to make it better
In my Docker Swarm clusters I just use a regular Apache2 image in front of everything, since mod_md is good enough for Let's Encrypt and it doesn't have the issue with "nginx: [emerg] host not found in upstream" that Nginx did when some containers are not available and are restarting (and none of that "nginx: [emerg] "proxy_redirect default" cannot be used with "proxy_pass" directive with variables" stuff either).
From the cases where I've used Kubernetes, the Nginx based ingress controller was pretty okay. I wonder why we never got Ingress Controllers for Kubernetes that are made with something like Apache2 under the hood, given how many people out there use it and how the implementation details tend to be useful to know anyways. Looking at the names in the new list of Gateway https://gateway-api.sigs.k8s.io/implementations/ in very much seems it's once more a case of NIH, although it's nice that LiteSpeed and Traefik and HAProxy are there.
28 comments
[ 2.8 ms ] story [ 52.6 ms ] threadIngresses with custom nginx attributes might be tricky to migrate.
And I do not understand it:
1. Ingress still works, it's not deprecated.
2. There a lot of controllers, which supports both: Gateway API and Ingress (for example Traefik)
So, how Ingress Nginx retiring related / affects switch to Gateway API?
In a way, it was a necessity, since Ingress was all you'd get and without stuff like rewriting, doing gradual Kubernetes migrations would have been much more difficult to impossible. For that reason, every ingress controller tried to go a similar, but distinctly different way, with vastly incompatible elements, failing to gain traction. In a way I'm thankful they didn't try to reimplement nginx annotations (apart from one attempt I think), since we would have been stuck with those for foreseeable future.
Gateway API is the next-gen standardized thing to do ingress, pluggable and upgradable without being bound to a Kubernetes version. It delivers _some_ of the most requested features for Ingress, extending on the ingress concept quite a bit. While there is also quite a bit of mental overhead and concepts only really needed by a handful of people, just getting everyone to use one concept is a big big win for the community.
Ingress might not be deprecated, but in a way it was late to the party back in the day (OpenShift still has Route objects from that era because ingress was missing) and has somewhat overstayed its welcome. You can redefine Ingress in terms of Gateway API and this is probably what all the implementers will do.
I do not understand.
Sad to see such a core component die, but I guess now everyone has to migrate to gateways.
Replacing an ingress controller in Kubernetes is also a well documented practice, with minimum or even zero downtime if you want to.
Generally, if your engineering team can reasonably keep things simple, it's good. However, business needs to grow and infrastructure needs to scale out. Sometimes trying too hard to be simple is, in my experience, how things become unmanageably complex.
I find well-engineered complexity to be much more pleasant to work with.
https://www.haproxy.com/blog/announcing-haproxy-unified-gate...
What is missing is an open source orchestrator that has a feature freeze and isn't Nomad or docker swarm.
It's untenable. In my own company we rely on several critical to us OSS projects and my appeals to donate to them are rebuffed :/
At least we try to patch stuff upstream instead of hoarding fixes, but I still think it's not just immoral but ultimately shortsighted.
They can ask Linux Foundation for money, they have plenty.
https://traefik.io/blog/transition-from-ingress-nginx-to-tra...
From the cases where I've used Kubernetes, the Nginx based ingress controller was pretty okay. I wonder why we never got Ingress Controllers for Kubernetes that are made with something like Apache2 under the hood, given how many people out there use it and how the implementation details tend to be useful to know anyways. Looking at the names in the new list of Gateway https://gateway-api.sigs.k8s.io/implementations/ in very much seems it's once more a case of NIH, although it's nice that LiteSpeed and Traefik and HAProxy are there.