HN Explorer

Ask HN: How to Deal with the Container Nightmare?

4 points by knowhy ↗ HN
I'm a Linux infrastructure engineer. I'm having a hard time in finding a position which doesn't sound like a total nightmare to begin with. All I see these days involves Kubernetes. I had my fair share of k8s and I'm not a fan. I'm not totally against it, there are legit use cases for it. But for the companies I interview with it almost always seems a bad choice and sign of a poor overall architecture and management.

Usually I'm confronted with something like this: an external developer or consulting agency has setup an initial container environment. And now the company is looking for someone to take over the environment. Usually nobody in the company has ever worked with Docker. All are full of high hopes and unrealistic expectations. Everyone thinks that the initial setup was the hard part and operating it will be an easy job. A lot of money and effort has been invested. The CTO got his job by pitching "let's decouple the monolith into modern micro-services", but there is only nihilism beyond that.

So much has been invested into this micro-service thing. The plan is perfect. Now I'm being asked to take over responsibility for this Frankenstein stack full of alpha quality software and good for now code. Nobody foresaw the heavy footprint of clustering everything. So there is no budget for a test environment.

I like KISS and boring code. Some ideas of DevOps are appealing to me but with the situation today it feels like going back to square one. Bringing up even the slightest critic of micro services leads to someone acting like repeating the GCP/AWS sales pitch would be an original idea.

Sometimes I stumble over comments here which give me glimpse of hope that some people are waking up and get a more realistic view on micro services. But I fear it will take a while until that will have an impact on my local job market. How long might that take? What can I do with this situation?

5 comments

[ 2.8 ms ] story [ 17.6 ms ] thread
square_usual
Two things here:

- Your job won't always involve doing things you like doing. If you're stuck in jobs which use tech you don't like, at some point you have to accept that said tech is here to stay and spend just your work time on it. It's not a great choice, but jobs are like that more often than not.

- If, as you say, containerisation is a bad idea for most companies (I wouldn't know - my company uses the "boring" way of deploying our services), you just need to point out how it's hindering them. I'm sure when management starts wondering why their platform bills are going through the roof or features take ages to go live, you can pitch the boring way to management and make your case with actual facts.

knowhy
Yes, it's true that at some point I have to stick with a tech I don't like. In general I don't mind doing that. At the end of the day I work for money. The question I'm asking myself is at which point that might be. Sadly when interviewing I'm not in the position to point out architectural pain points. It feels impossible to argue against k8s. Every time I tried that I get some unreflected cargo cult answer. The people I interview with already invested in k8s so it seems pointless to indicate that there are some massive downsides.
p0d
I left the IT industry last year and have found myself in the education sector. I can't speak into your future but I do recollect sharing some of your frustrations as a sysadmin of 20 years. k8s felt to me like whack a mole. I was old school. I created relationships with the right people in our datacenters. My team shipped and deployed our own bare metal servers. Ansible was great for deploying VMs. Then came along k8s.

I was happy with our freebsd servers doing zfs and nfs. I was a fan of nginx as a proxy server on linux. I got redis. Mongo came and went. It wasn't rocket science to put failover services on different hardware and in different datacenters.

I would rather have a collection of small parts making up a complex solution than the complexity of k8s.

I have gotten a lot of mileage out of freebsd jails and lxc on ubuntu down through the years. I am a fan of containers. I agree with you that people use k8s for deployments which really don't need it. Unfortunately this is the world of IT; new toys and, "na, na, I know something you don't".

I moved to education as I decided I didn't want to be responsible for an IT infrastructure anymore. My new job is equally difficult but does not require the same out of hours headspace as looking after a global network.

knowhy
> I would rather have a collection of small parts making up a complex solution than the complexity of k8s.

I feel the same way. If I wouldn't have stumbled over Ansible 6-7 years ago I would have left the industry already. I don't mind complexity in general but I feel it should be in line with company needs. And k8s often isn't.

You are right, FreeBSD jails, lxc, etc are great technologies. It's k8s and related topis I have my issues with.

Do you regret leaving the industry?

I have invested a lot of effort into my career and it would feel kind of stupid to give up on it because I have some issues with today's industry trend. On the other hand you are right, k8s is the technical side of a people problem in the industry. While I'm hopeful that things around k8s will settle on the long run, I'm pessimistic that the cargo cult and believe in easy answers to complex problems will disappear any time soon.

p0d
I am happy to have left the industry but would not have done so over an IT fad. I left as managing an IT infrastructure takes up a lot of headspace. Even if you aren't working all the time stuff is on your mind. I'm very happy in the education sector now.