> Knative is about to hit v1.0 and become «stable», has a solid community.
I'm sorry for the negativity but I'm skeptical about the reliability.
AFAIK it still doesn't really address instabilities. Software crashes, hardware fails, power goes out. It is trivial to lose data, and it is trivial to be blissfully unaware that there is a possibility of data loss because when everything's green, things work perfectly fine.
My anecdote is, I was at a conference, listening to the "Introduction to Knative" talk. Knew just the name and that serverless is a hot new thing. There was a simple app that has sourced earthquakes and displayed a map, or something like that. Things looked neat and simple, so I wanted to jump on that bandwagon, but... My first (and quite obvious) question was, "what happens if during the event processing a hardware node that runs a service instance suddenly goes dark?".
I was surprised that there wasn't a meaningful answer, so I've tried to research it myself and found that - in my understanding - the event is just lost. Unless someone had taken extra care to implement such guarantees by adding more and more statefulness. As I get it, it's still K8s, and I can deploy my own message bus/queue, but this devalues Knative for me.
I am not sure if an event queue is the primary focus of Knative, or at least this particular blog post. This is focusing on the problem of Kubernetes networking, which is honestly pretty useless.
By default, a Kubernetes "service" exists to create a single IP address for a group of replicas. You see that your "foo service" is at 10.2.3.4, you open a TCP connection to it, your connection is created to one of the available replicas. You open another connection, it goes to another replica. This accomplishes some load balancing and lets new TCP connections avoid unhealthy replicas.
The problem is, a TCP connection per request is relatively uncommon these days. You open up one connection to MySQL and send it multiple queries. gRPC sends multiple requests and responses with one TCP connection. HTTP/2 sends multiple requests with one TCP connection. The days of one request = one TCP connection are over.
So there is a lot of work being done to try to make load balancing transparent at a level above the TCP connection. Your app asks to connect to foo-service, and each request is routed to another healthy replica. Replicas come and go, but your app never has to handle reconnects. You want to send 10% of requests to a canary instance without having to tell your application that; you want the networking layer to handle that for you.
There are several approaches to handling this problem.
The first is having smart clients; if your gRPC library knows how to ask the Kubernetes API for a list of replicas, it can do its own load balancing and health checks. It gets a list of valid endpoints, opens up a connection, subscribes to the streaming health service that each backend provides, decides on a load-balancing algorithm, and goes! This works quite well, but it is difficult to setup and difficult to maintain. The C++ gRPC client has to do this, the C++ HTTP/2 client has to do this, the Go gRPC client has to do this, the Go HTTP/2 client has to do this, the Python gRPC client has to do this, the Python HTTP/2 client has to do this, the node.js gRPC client has to do this... you get the picture.
I am most familiar with this approach from my days at Google, where load balancing typically consisted of a smart client for every language that connected to a coordinator, asked for backends and a rule for splitting traffic among them, and stayed connected to receive updates. It worked extremely well. You could drain a cluster very quickly, and obviously traffic kept flowing if the load balancer died. (Not sure I ever saw the load balancer die, though.) The disadvantage is that a malfunctioning smart client can break everything; every piece of code you deploy to production in every language has to work perfectly. (From reading job ads, I feel like few companies are as disciplined as Google at saying "you have to use one of these 3 supported programming languages". So in the real world, the "smart client" approach will never work. You can check out the existing gRPC client libraries to see this in action; best case, there is round-robin load balancing based on DNS records, with some metadata that can also be retrieved from DNS. C++ supports it best, Go is next, the rest of the language bindings... questionable. Raw HTTP/2? Good luck.)
The second approach is to have a smart proxy that your apps use to get to the outside world. You run this smart proxy as a "sidecar", receive all traffic through it, and send all outgoing traffic through it. It knows where all the backends are and how to load balance, so the app doesn't have to. Personally, I think this is the most practical approach right now. Run Envoy next to your apps, make your services "headless" (so Kubernetes just returns all endpoints over DNS, rather than trying to do TCP-level load balancing), and go! Even your outdated PHP app will now have detailed metrics and participate in distributed tracing. The disadvantage is that you have to manually add an Envoy co...
I normally don't leave "thank you" comments since they clutter up the discussion, but your comment above was particularly valuable to me and I wanted to make sure you knew that. Thank you :-)
> My personal opinion is that they do too much; if you name your syslog port "grpc" by accident, goodbye traffic! What was supposed to be documentation is now just an undocumented part of the Kubernetes API.
To be fair, a LOT of things in the Kubernetes world can rely on port names. Service and Ingress resources can use them to target ports on pods (or services, in the case of Ingress resources). Port names are responsible for the name of DNS SRV protocol specifiers for headless services as well.
>The disadvantage is that you have to manually add an Envoy container and minimal config to everything you run, which people apparently hate.
Correct me if I'm wrong, but I think envoy (or istio, at least), uses a mutable admission webhook to inject the sidecar into your pod. This means you never have to enter anything into your pod spec.
What you're describing with injecting outbound proxies is definitely something that Istio does in service mesh mode. When Knative launched 0.1 over a year ago, we had a pretty hard dependency on Istio. We heard a lot of feedback from users who weren't ready to make that much commitment, and as of 0.3 (last December or so), Knative supports multiple pluggable HTTP routers, including Gloo, Ambassador and Kourier.
If you aren't using Istio in mesh mode, Knative only controls inbound HTTP networking to your container instances, which it uses to implement scale-from-zero, concurrency-based-balancing, and percentage based version rollout. Outbound network is completely unaffected, except that containers may be shut down when they are no longer handing a request.
Still waiting for it to support basic, but crucial functionality like tolerations and node affinities. And no, using admission plugins like PodNodeSelector is not viable on managed clusters like GKE.
That seem premature and unfair. KNative has representation from Google, IBM and Redhat, and the steering committee is working on a more inclusive governance structure.
(I have no inside knowledge of this - just what I have gleaned from slack ;-) )
Representation is based on contributions to the project, and Google has publicly stated that they look forward to not being a majority on the steering committee (when other companies have exceeded Google's contributions).
There are CNCF projects whose governance is entirely within a single company. I get that there's a great narrative here that "Google doesn't share", but it seems like Google is being held to a higher bar here.
17 comments
[ 2.3 ms ] story [ 54.4 ms ] threadI'm sorry for the negativity but I'm skeptical about the reliability.
AFAIK it still doesn't really address instabilities. Software crashes, hardware fails, power goes out. It is trivial to lose data, and it is trivial to be blissfully unaware that there is a possibility of data loss because when everything's green, things work perfectly fine.
My anecdote is, I was at a conference, listening to the "Introduction to Knative" talk. Knew just the name and that serverless is a hot new thing. There was a simple app that has sourced earthquakes and displayed a map, or something like that. Things looked neat and simple, so I wanted to jump on that bandwagon, but... My first (and quite obvious) question was, "what happens if during the event processing a hardware node that runs a service instance suddenly goes dark?".
I was surprised that there wasn't a meaningful answer, so I've tried to research it myself and found that - in my understanding - the event is just lost. Unless someone had taken extra care to implement such guarantees by adding more and more statefulness. As I get it, it's still K8s, and I can deploy my own message bus/queue, but this devalues Knative for me.
https://github.com/knative/eventing-contrib/issues/656 is the issue that tracks it, and it was shoved away to the eventing-contrib repo...
It could be that I don't understand something, or that I've made the wrong conclusions.
---
Update: Found https://github.com/knative/eventing/pull/1949 - seems that they've merged in something just yesterday. Seems that things are improving. That's good.
Think of it as an open source lambda that runs on top of Kubernetes.
I like Google's strategy here: Cloud run is (essentially) hosted Knative for those that want to pay someone else to manage it.
For DIY, you can run it on your own cluster.
By default, a Kubernetes "service" exists to create a single IP address for a group of replicas. You see that your "foo service" is at 10.2.3.4, you open a TCP connection to it, your connection is created to one of the available replicas. You open another connection, it goes to another replica. This accomplishes some load balancing and lets new TCP connections avoid unhealthy replicas.
The problem is, a TCP connection per request is relatively uncommon these days. You open up one connection to MySQL and send it multiple queries. gRPC sends multiple requests and responses with one TCP connection. HTTP/2 sends multiple requests with one TCP connection. The days of one request = one TCP connection are over.
So there is a lot of work being done to try to make load balancing transparent at a level above the TCP connection. Your app asks to connect to foo-service, and each request is routed to another healthy replica. Replicas come and go, but your app never has to handle reconnects. You want to send 10% of requests to a canary instance without having to tell your application that; you want the networking layer to handle that for you.
There are several approaches to handling this problem.
The first is having smart clients; if your gRPC library knows how to ask the Kubernetes API for a list of replicas, it can do its own load balancing and health checks. It gets a list of valid endpoints, opens up a connection, subscribes to the streaming health service that each backend provides, decides on a load-balancing algorithm, and goes! This works quite well, but it is difficult to setup and difficult to maintain. The C++ gRPC client has to do this, the C++ HTTP/2 client has to do this, the Go gRPC client has to do this, the Go HTTP/2 client has to do this, the Python gRPC client has to do this, the Python HTTP/2 client has to do this, the node.js gRPC client has to do this... you get the picture.
I am most familiar with this approach from my days at Google, where load balancing typically consisted of a smart client for every language that connected to a coordinator, asked for backends and a rule for splitting traffic among them, and stayed connected to receive updates. It worked extremely well. You could drain a cluster very quickly, and obviously traffic kept flowing if the load balancer died. (Not sure I ever saw the load balancer die, though.) The disadvantage is that a malfunctioning smart client can break everything; every piece of code you deploy to production in every language has to work perfectly. (From reading job ads, I feel like few companies are as disciplined as Google at saying "you have to use one of these 3 supported programming languages". So in the real world, the "smart client" approach will never work. You can check out the existing gRPC client libraries to see this in action; best case, there is round-robin load balancing based on DNS records, with some metadata that can also be retrieved from DNS. C++ supports it best, Go is next, the rest of the language bindings... questionable. Raw HTTP/2? Good luck.)
The second approach is to have a smart proxy that your apps use to get to the outside world. You run this smart proxy as a "sidecar", receive all traffic through it, and send all outgoing traffic through it. It knows where all the backends are and how to load balance, so the app doesn't have to. Personally, I think this is the most practical approach right now. Run Envoy next to your apps, make your services "headless" (so Kubernetes just returns all endpoints over DNS, rather than trying to do TCP-level load balancing), and go! Even your outdated PHP app will now have detailed metrics and participate in distributed tracing. The disadvantage is that you have to manually add an Envoy co...
To be fair, a LOT of things in the Kubernetes world can rely on port names. Service and Ingress resources can use them to target ports on pods (or services, in the case of Ingress resources). Port names are responsible for the name of DNS SRV protocol specifiers for headless services as well.
Correct me if I'm wrong, but I think envoy (or istio, at least), uses a mutable admission webhook to inject the sidecar into your pod. This means you never have to enter anything into your pod spec.
Edit: I noticed you pointed this out with istio.
What you're describing with injecting outbound proxies is definitely something that Istio does in service mesh mode. When Knative launched 0.1 over a year ago, we had a pretty hard dependency on Istio. We heard a lot of feedback from users who weren't ready to make that much commitment, and as of 0.3 (last December or so), Knative supports multiple pluggable HTTP routers, including Gloo, Ambassador and Kourier.
If you aren't using Istio in mesh mode, Knative only controls inbound HTTP networking to your container instances, which it uses to implement scale-from-zero, concurrency-based-balancing, and percentage based version rollout. Outbound network is completely unaffected, except that containers may be shut down when they are no longer handing a request.
Can we stop stacking shit?
And why is it unfair to ask for open governance from companies other than Google and IBM (redhat is part of IBM)?
Representation is based on contributions to the project, and Google has publicly stated that they look forward to not being a majority on the steering committee (when other companies have exceeded Google's contributions).
There are CNCF projects whose governance is entirely within a single company. I get that there's a great narrative here that "Google doesn't share", but it seems like Google is being held to a higher bar here.