Ask HN: Is one-man-devops a realistic project?
Our system/ops guys (~50 people) are generally smart and nice people - but they're the classic victim Ops dudes that got renamed to DevOps overnight. They went from scp'ing bash scripts to bare-metal to AWS and Kuberenetes and all that jazz and are basically "winging it". The rest of the frim (few hundred devs, business/marketing/support) are either blissfully unaware or getting increasingly frustrated as deployments remain a coinflip for success.
I am (or was) the dev tech lead here, and recently quit mostly due to some of the frustrations mentioned above. I've already secured an offer from "tech giant X", and intended to move in a few days. As a "counter" from the CEO - I've have the option of creating a new division, with the initial goal of redoing the entire stack.
I have a free hand in hiring/strategy, except from within the company. It's super hard to recruit here due to location and strict security requirements, meaning that for 6+ months I'd be "alone".
The goal is a production-grade Kub cluster on AWS with the accompanying ecosystem. That includes most of the SOA-enablers - full CI/CD, multiple segregated envs, monitoring/logging, service catalog, etc. When it's "ready", we handover the day-to-day "back" to ops. My division goes on to bigger and better things - whatever I'm into.
I have a very solid grasp on things required on the dev side, but the overall AWS architecture (networking, security, etc) is not something I've dealt with. Some things can be handed over to managed solutions (e.g. KMS), but strict regulation means not all. EKS/Tectonic and such are probably not an option.
Those of you with this kind of transformation behind them - is this a realistic goal for 1-2 non-ops people to learn and an implement within ~1 year?
5 comments
[ 12.7 ms ] story [ 795 ms ] threadHowever, "production-grade" is a term more easily thrown around than implemented, especially in the financial sector. I assume hiring consultants from that is not an option?
That's just a name change. "DevOps" is just the same system administration as it was for the previous 20 years, the difference is just that now we have more tooling, services, and deployment methods. Why "victim dudes"?
> They went from scp'ing bash scripts to bare-metal to AWS and Kuberenetes and all that jazz and are basically "winging it".
OK, so they got virtual machines, somebody else's data storage and processing services, and tooling for containers suddenly dropped on their lap, so they need to modernize their craft. And from what you describe, they weren't that good in the craft to begin with (IaaC is an idea twenty years old, the same with packaging software, and just these two techniques can easily handle growth from a few machines to dozens or hundreds).
> [...] is this a realistic goal for 1-2 non-ops people to learn and an implement within ~1 year?
Probably not. For building a robust infrastructure you need somebody who understands well how the OS and networks work and what mechanisms the OS provides, which usually means a good and experienced sysadmin, not a programmer. It's not impossible for a programmer to learn that, but it requires a good mentor, and then you're still left with an implementation.
One should be someone with a deep understanding of the bare metal networking, security, etc. concerns, who's enthusiastic about learning how that translates to an AWS environment. S/he will advise you to make sure you get that part right, and hopefully pick up enough to pitch in to design and build your infrastructure.
The other should be someone with a strong understanding of the old bash scripting who's interested in learning modern devops scripting. They'll help you translate the old scripts to new paradigms and eventually help you build out the rest.
Both of these people will have to be fast learners willing to spend the first month or so learning the essentials. Make sure you have funding from management to support whatever video tutorials, online courses, etc. they need to get up to speed.
If I were you, the question I would ask is do I want to go through this struggle? Would I enjoy it? DevOps has it's own frustration moments (little more than programming) & Aha moments. Not everyone enjoys it, so ask yourself that question first.