Ask HN: What do you think will come after Kubernetes?
I mean, Kubernetes became a thing after Docker became a thing before(and so on, all the way down to VMs), so it's reasonable to assume that something new will eventually emerge and become the industry-standard.
175 comments
[ 2.8 ms ] story [ 163 ms ] threadBy that I mean, entirely API driven, declarative, etc.
What we need is an abstract layer to automate the systems underneath so we can leverage that abstraction and create more complexity
There are still some folks who understand the abstraction all the way up and down. We cant have that
From my point of view 80-90% of software people doesn't need k8s really.
There is strong trend to have "low code-no code" at some point we want abstraction of CI/CD and having small apps that can be built by specialists without need for developers. With k8s managed by cloud vendors we are in the middle of "no infra", cloud vendors will be managing k8s clusters but it is not going to be that everyone wants to spin up his own cluster. There is not enough market for higher infra abstraction, there is enough market for level where we are at.
The big win of kubernetes is not that it lets you get access to computational resources easier (yaml rather than systemd scripts) but that it is a temporary escape on the "file a ticket to get new compute resources" (temporary because just like picking up hardware for the data center goes from just buy it to get it approved; VM creation goes from just click a button to what's your cost center and VP signoff; new namespace creation hasn't had time to get put into service now, but it will be). Like Docker let developers bypass restrictive OS imaging, k8s lets developers spin up more resources with less constraint. It's not really about the technical abstractions, running a go binary on bare metal is not different (only faster) than running it in a k8s cluster except I have no way to get permission to run it on bare metal.
And my skills are shifting from "maximizing the product value delivered by this heap of hardware" to "minimizing the cost of the MVP in this cloud billing environment." Before if we had unused compute, we'd stick a cache in or precompute something that will reduce latency for customers down the line. Now, we are like, don't do that computation, it mightn't be needed.
Keep building those layers... thats job security
There's been a few popping up recently, time will tell who wins
and Porter https://news.ycombinator.com/item?id=26993421
were the ones I've seen recently, although admittedly on re-look I may have misunderstood Space Cloud and I could have sworn there was a third
[1] https://convox.com/
Had some fun a few years ago with toy projects. Can't tell its current state...
I can't wait for the next layer of complexity to be added. It will no doubt be called something like WizzlyBobATron.
Tron?
The Master Control Program?
I think this will be driven by a few trends:
- Very fast disks to store data SQLite style, auto replicated transparently at the disk level.
- Typed languages becoming more usable (tooling and language improvements), which makes it easier to design and operate systems (stack traces vs distributed debugging/tracing).
- Improvements in Heroku-like systems from AWS/GCP etc.
With kafka as the data source every instance was autonomous and would keep itself up to date using the kafka partition offsets.
Went really well, took the DB cluster out of the scaling equasion and offloaded a lot of the cost to the SSD instead of memory or a managed DB Cluster.
If eventual consistency is enough i feel that this is a nice way to handle things.
I find a bit complicated using streams / queues for everything because it is a paradigm shift (turning the database inside out).
BUT when you have a clear use case like ingestion -> refinement step 1-> refinement step 2 it is a _very_ natural way to do stuff with the added benefit I can have as many parallel workers as topic partitions.
Frankly I think the serverless model will end up dominating in the long run. Whats really missing there isnt a better abstraction for k8s. Its the dx workflow and local development experience that still requires lots of work. Emulators for services, workflows, good ide integration, open service standard. That will all be where the next layer happens.
Developers need to be able to run the real platform, like they can with k8s/k3s.
I’m happy with compose. It works. I know swarm was supposed to solve this but is dead now (?).
One way to do it is you have semi-imperative code that runs, the output of the code is a description of the system to be deployed. Then you have some kind of diffing system that figures out how to take your existing cloud deployment and turn it into the new version described by the output of your code.
This is how Pulumi works for example.
No dashboard where you can muck about, no shared account that mixes together resources from all your environments, no messy state managment errors because you are always working with the same state: the truth of what is running right now.
The console would only be for monitoring, observability and maybe some disaster recovery actions.
The idea of infrastructure as code in relation to Kubernetes is that you have the full power of a programming language to build with, not just yaml files. You can also tie into other functionality of cloud providers as long as your IAC provider supports it. So you could have your Kubernetes cluster connect to some serverless code, or to a managed database, all deployed from one codebase.
Having said all this I didn’t actually have a great experience using Pulumi and switched to plain K8s. It makes a ton of sense in theory though and I’ll probably try again.
For more simple workflows, e.g. a single API endpoint, there's serverless and other SaaS services out there which will let you build off from that, and lets you save a huge amount of time and money compared to building it traditionally with a Web framework.
Now that we've had a few years of experience with how people are actually using the platform , I could see a simplified version of Kubernetes taking hold for businesses still running complex sets of services, something that's easy to install, actually comes with batteries included and is secure by default.
The decision to go with GKE saved me enough money to cover payroll and then some compared to Heroku.
-Easier to auto-scale because it does not have any state
-Can easily run at the Edge (look into lambda at edge of cloudflare worker)
- Zero to very minimal devops / syadmin knowledge required. - Monitoring and few other important tooling is usually out of the box
- Add on / Related services in serverless stacks like vercel/netlify etc also handles email/auth type components, a framework will only provider library for these, implementation is in developer hands.
- Simplified deployment : most services can sync with Git repos, provide one click deploy and have simple CLI binaries .
Docker with k8s is awful when it comes to resource metering. Docker is terrible with security and resource isolation. K8s, in general, is awful IMO because of the way it was architected and implemented.
Xen-ified "containers" (regular VMs but managed) with an ability to inject uniform, templated, configured concerns is how to do services: micro or macro; ephemeral app, infrastructure, or backend. Also, backups and logs have to live somewhere else.
Basically, a APPaaS/PaaS/IaaS in the spirit of K8s + OpenStack but much cleaner, workable, easier, simpler, fewer choices, more usable, more reliable, less stupid, and more standardized. (Discovery is insecure and fundamentally unworkable; top-down provisioning knows exactly what exists and is alive so use that as the authoritative single-source-of-truth.)
There is no other way I can see.
You can say this about 80% of tech yet here we are.
If you don't understand how to build what's underneath, then you don't understand what's involved and are at the mercy of vendors or someone else.
Everybody has at best little or shaky knowledge of underlying abstraction. Yes it helps to know more, but is not really practical to expect that knowledge as the minimum required.
In my relatively short but rich career, I rarely came across a problem that was caused by the tech. Be it k8s or PHP, the problem was ppl using PHP while thinking in terms of Go/Python, or using Kubernetes for stateful monoliths, etc.
We blame the tech because we don’t spent time to understand how it works. So we use the next tool very happily until we come across a new design pattern, context, etc. The circle repeats...
Micro services have very well documented pros and cons. Monoliths too. At this point it’s all about the engineers using the tool at hand properly.
However what seems to be missed in a lot of assessments is what I’ve started calling “flying buttress architecture”… you have your central monolithic structure, but rising around it are dozens of smaller supporting elements, not the same as having a seconds task queue system, this is building these components as stand alone parts. These “flying buttresses” can be spun up and down ephemerally in a Kubernetes environment, built as a light weight services or as “run once” jobs and cronjobs scheduled by Kubernetes to use up the spare resources left over in each Kubernetes node around the main monolithic applications.
This makes Kubernetes more useful as a deployment and ops tool when dealing with traditional or existing/legacy software.
Sure, yaml is a pita in both of them.
For the next 10 years, I don’t see kubernetes going away but rather evolving into something that’s dead simple for anyone to setup and use easily. It’s flexibility in being a platform capable of easily running whatever you want to is somewhat unmatched.
Honestly the only way I see k8s being replaced is by a fork if there is some Governance dispute. I don’t see the platform changing for at least 10-20 years.
This is true but is that such a huge deal? We successfully moved from AWS to GCP and it wasn't a massive deal; just different terminology for the same concepts.
Compare that to redeploying your k8s manifests from an EKS cluster to a GKE cluster or vice versa. Its definitely not fully platform agnostic, but it requires much less engineering effort.
I know there are great terraform modules out there already, but i wonder if many of Hashicorp's products soon will be available as "managed" services as well, allowing people to try something new with less maintenance cost.
One company I worked with went managed, another one didn't. I think hashicorp stuff is too barebone in general. I noticed way less headaches with the managed solution and I would just host it with them.
In contrast, I feel comfortable running a k8s cluster for a smaller business of mine.
It's a work in progress but might be helpful to someone!
K8s in my book tries to solve arbitrarily complex deployment problems. To do so, K8s overall accepts being a complex monster. You have to fully commit to use K8s, you have to put everything you have into K8s, and then K8s solves a staggering amount of problems.
Nomad is much simpler. A good way I've found to describe nomad is: Nomad+Consul+Vault is essentially a distributed initd/SystemD + cron + secured store. That's all. It's powerful, because there are no problems you can't solve by scheduling + discovering VMs, containers and applications. But it's at a more fundamental and lower level, and it requires more work to utilize well. It's also less of a jump compared to K8s, but some stuff more or less solved in K8s is not in Nomad, so far.
I strongly enjoy working with nomad and we see other operations teams grow interested, because nomad functions and handles similar to something like VMWare. Sure, containers are weird, but it can do VMs, and it's still similar. For example, for an ES cluster on hardware with grafana/kibana/2 logstashes running "somewhere on these hosts", nomad is very pleasant to use.
But I doubt nomad will be as hyped as K8s because e.g. you cannot throw a helmchart at it to create a postgres/patroni cluster on abstracted storage for a servide with a relational storage requirement, as the hyperabstracted way needs.
CSI support exists and is being enhanced, but you will still need a CSI provider, and e.g. the GlusterFS CSI provider got eaten by K8s and now you're stuck with S3 or running Ceph. And running Ceph is a ... thing. And for example, you'll have to run the nomad autoscaler besides it, and other orchestration and management systems around it that K8s has, or has integrations for. There's just a lot of edges to handle, if you could just buy hosted K8s.
The benefit is huge though - you can orchestrate VMs and processes in addition to containers
if the lesson from k8s is that after containers we needed container orchestration - the business logic analog is that services will need service orchestration.
(disclaimer: i work here so i'm both biased and have skin in the game)
And perhaps geographical regulations will become so demanding that you need something like durable objects to store & process data in country of origin.
Disclaimer: I have not Kubernetes experience, show a few documents and opted for Nomad with great success so far.
Why don't do the same for a typical Kubernetes setup with Hasura, API server, Gateway, load balancer, Postgresql, Redis, and a couple of services for logs and monitoring all in one, with all the networking stuff already setup? Just give me an image or template made by someone else, maintained on Github, and ready to customise and develop for my needs.
The support guys had a complicated procedure for restarting it that involved a very carefully orchestrated sequence, which sometimes didn't work so they had to start from scratch.
It was awful to watch.
They didn’t try to run it all together until it was time to ship.
Networking overhead definitely killed us. That, and incompetence.