- This implementation is a good example of why Kubernetes ingress has been in beta for 2 years ;-). Ingress doesn't do very much, so the Kong team has had to implement a custom set of extensions to Ingress ("KongIngress") so you get functionality such as rate limiting, etc. This blog post https://blog.getambassador.io/kubernetes-ingress-nodeport-lo... covers some of the nuances of this.
- I wonder how the REST APIs of Kong are reconciled with the declarative config. Seems like this could be complicated.
- IIRC, all of the NGINX ingress controllers take the route of custom annotations on the ingress object vs the KongIngress approach. I wonder what people will prefer.
- The post alludes to intelligent connection draining with Kong but doesn't give any details. I'm particularly curious because I thought Kong was based on NGINX so wondering how Kong does it given the underlying engine.
Do we know what the Kubernetes' team's official plan for fixing the ingress situation is?
My opinion is that ingress portability isn't really that useful in practice. You want implementation lock-in at the ingress level, since the implementation matters. If I need to target platforms in a generic way (e.g. make something that works on both AWS and GKE), I can use Helm or some other templating solution to generate the right manifests with the right settings.
So "KongIngress" isn't a problem for me, and rather than struggle with today's situation on GKE, I'd much rather have a "GoogleIngress" with all the GLBC bells and whistles (well, there aren't many, but at least timeouts and a CDN toggle!) and not care about it being portable.
At KubeCon Austin in December there was a lot of conversation about it. Some want portability, some want expressiveness, everyone agrees that current ingress isn't expressive enough.
In practice, everyone adds custom annotations one way or another (whether it's NGINX ingress, or Ambassador, or KongIngress). So at some level, the notion of Kube ingress is a moot point, since everyone has forked it :-).
I kind of agree. At this point it may just be better to write controllers that read generic policy from a different object and the base definition from the Kube object. But at that point, the extra ingress object is just a convenience.
As a frequent user of ingress, I agree that portability isn't useful in practice and that requirement is a hindrance on ingress evolution. A lot of work has been put into CRDs, yet parts of the community push for a "portable" ingress. With all the one-off features/settings, users will inevitably make substantial changes to their ingress object - might as well use CRDs with clear specs. The alternative is annotation hell or associated configmaps.
I've heard some say they want to be able to list out all external addresses exposed from the cluster, but that can be solved with a new resource more akin to endpoints.
I'm referring to Cloud CDN [1], which is per-backend setting that is disabled by default.
Unlike some other offerings (like AWS CloudFront), the CDN is transparent and built into the GLBC. Just by turning on the CDN, your load balancer automatically gets a globally distributed cache.
The reason we didn't adopt Kubernetes Ingresses for Ambassador (https://getambassador.io) is more or less the same reason you cite in your first bullet. The interface is too simple and while it may work for the simple case of configuring a reverse proxy it rapidly falls apart in the face of things like advanced API capabilities such as rate limiting, shadow traffic, canaries etc. It turns out these are things people really care about.
This also requires a datastore such as Postgres or Cassandra instead of just using the kubernetes api via a CRD or something else. Makes it a bit of a downer to use.
This is great. I currently do the samething but not rely on Ingress but rely on label and annotation of service.
Plugin config is also annotation.
It's a bit complex to make plugin concept work because Kong doesn't support some kind of idempotent request so lots of code is require to check for exist and set.
Such as when a service is updated, have to check/unset plugin and reupdate plugin config etc.
I've had great success with Kong acting as an API gateway running on a DC/OS cluster. It was simple to work with and it's pairing with Cassandra as a backend was an added bonus for multi region and availability zone setups.
I'm looking forward to giving this a shot on k8s. Nice work KongHQ team!
It's hard to tell what you get from deploying Kong as an Ingress vs as a Deployment and Service pair. Is it just a declarative config process rather than a REST API one?
15 comments
[ 3.6 ms ] story [ 45.8 ms ] thread- This implementation is a good example of why Kubernetes ingress has been in beta for 2 years ;-). Ingress doesn't do very much, so the Kong team has had to implement a custom set of extensions to Ingress ("KongIngress") so you get functionality such as rate limiting, etc. This blog post https://blog.getambassador.io/kubernetes-ingress-nodeport-lo... covers some of the nuances of this.
- I wonder how the REST APIs of Kong are reconciled with the declarative config. Seems like this could be complicated.
- IIRC, all of the NGINX ingress controllers take the route of custom annotations on the ingress object vs the KongIngress approach. I wonder what people will prefer.
- The post alludes to intelligent connection draining with Kong but doesn't give any details. I'm particularly curious because I thought Kong was based on NGINX so wondering how Kong does it given the underlying engine.
Coming from nginx/apache rulesets where you can pretty much route a request however you want to a simple Ingress is jarring.
My opinion is that ingress portability isn't really that useful in practice. You want implementation lock-in at the ingress level, since the implementation matters. If I need to target platforms in a generic way (e.g. make something that works on both AWS and GKE), I can use Helm or some other templating solution to generate the right manifests with the right settings.
So "KongIngress" isn't a problem for me, and rather than struggle with today's situation on GKE, I'd much rather have a "GoogleIngress" with all the GLBC bells and whistles (well, there aren't many, but at least timeouts and a CDN toggle!) and not care about it being portable.
In practice, everyone adds custom annotations one way or another (whether it's NGINX ingress, or Ambassador, or KongIngress). So at some level, the notion of Kube ingress is a moot point, since everyone has forked it :-).
I've heard some say they want to be able to list out all external addresses exposed from the cluster, but that can be solved with a new resource more akin to endpoints.
Hmm, what would this do exactly?
Unlike some other offerings (like AWS CloudFront), the CDN is transparent and built into the GLBC. Just by turning on the CDN, your load balancer automatically gets a globally distributed cache.
[1] https://cloud.google.com/cdn/
A generic way to delegate to an arbitrary CDN might be nice too.
Plugin config is also annotation.
It's a bit complex to make plugin concept work because Kong doesn't support some kind of idempotent request so lots of code is require to check for exist and set.
Such as when a service is updated, have to check/unset plugin and reupdate plugin config etc.
Great to see Kong made this step.
I'm looking forward to giving this a shot on k8s. Nice work KongHQ team!