Ask HN: Why do I exist as a sysadmin?
As more and more services and tools are rolled into cloud provider's portfolios, I can't help but think that there's no point for me to exist. Here on HN, in person with other tech-minded people that I know, and elsewhere in the tech sphere, I'm bombarded with a viewpoint that boils down to: "Be a developer, or get out of the tech industry."
Herein lies the issue. I have thus far been unsuccessful in any programming endeavors that I have attempted. I can manage to throw together bash scripts and other glue that's necessary to make automation tasks function at my work, but these are largely cargo culted from various stack overflow posts or other online resources. Whenever I attempt to dive into anything that I would consider "development", such as python programming, it's as if my brain completely ceases to function and all of the words and symbols on my screen turn into an incomprehensible alien language. It doesn't matter how long I stare at it or how much I reference the documentation, that alien language never reveals its true meaning to me.
This inability to understand syntax is not only saddening to me, it's made much worse by a variety of mental issues that seem to amplify the issue, such as: PTSD, anxiety, depression, borderline personality disorder, and dyscalculia. All of these feed off of each other in a vicious cycle of frustration, hopelessness, self-loathing, and worse.
I suppose my question is:
Has anyone had a similar experience and managed to push through the mental blocks associated with learning to program while having your own mind constantly working against you?
71 comments
[ 3.3 ms ] story [ 113 ms ] threadOr rather, the experience of an SA should translate decently into SE
Okay, now to the coding part.
"Has anyone had a similar experience and managed to push through the mental blocks associated with learning to program while having your own mind constantly working against you?"
All the time. I have the depression, and a few other mental problems most likely, and I have to say, for as much as my brain helps me it hurts me a lot too. Especially with programming and engineering, I feel like things are impossible right up until there's some breakthrough, and then things are fine. It happens really quickly, and soon you're just onto the next problem - many times I feel I don't relish the victory of whatever I have solved enough before moving onto the next thing. You might be doing the same thing too - I can't tell how much or little progress you've made, but learning is hard, and it makes everyone feel dumb, not just you.
"I can manage to throw together bash scripts and other glue that's necessary to make automation tasks function at my work, but these are largely cargo culted from various stack overflow posts or other online resources" This is more of the job than you'd think. I honestly think you're on the right track here.
As a sysadmin, are you good at debugging other people's code when something breaks (do you try? sometimes there's org boundaries that prevent this)? That is also a hugely valuable skill. I know some people who are frightened of writing code fresh, but are great at finding and fixing bugs.
I'm not sure this will be of much help, but I also think as a society we're all about fixing our problems rather than leveraging our strengths. If coding just doesn't click with you, that's fine, obviously other tech things do click with you. Keep working on finding what works for you.
As far as debugging other people's work, I do that a fair bit within our configuration management system. I also write "fresh" config management states (we use SaltStack), fairly often as well. But SaltStack is mostly human-readable YAML/Jinja, so it doesn't seem to cause the same issues as trying to write and parse Python.
This is the first time that I have heard or read anyone say this in years. Thanks
Best of luck with you endeavors, my sympathy for your struggles.
...This is bull----, and it probably always will be. There is a vastly higher demand for implementation and support and security than there will ever be for people to create the stuff in the first place. While learning to code is a big plus (I am a hobbyist developer when my caffeine is flowing enough for me to process code), there will always be a huge need for people who understand the priorities and needs of the companies they work for who can recommend, assemble, and manage solutions for them. Even in a fully cloud-based setup, your average company will still need someone who understands the cloud and the applications operating on it to support them and ensure they aren't being taken advantage of.
Sure, plenty of IT jobs are getting shuffled off to blue collar outsourcing shops, but check out stories about Wipro this week to understand why that is going to backfire significantly in the long run. Our job is not just to make it work, but to anticipate problems and provide affordable, workable solutions that mitigate the risks of those we work for.
Thank you!!
...Most people who move to the cloud end up still needing IT staff to manage it, and are now, by proxy, paying for the cloud provider's IT staff too. (Hidden underneath all of those cloud providers are a silent army of IT people.)
A recent conversation I had was whether to replace a server (five figures) when a neighboring organization was on the cloud for the same technology (three figures per year per device). The latter sounded a lot cheaper until you multiplied the devices by that three figure amount, and multiplied it by the five years the server's contract covered. The cloud solution became more expensive that the physical service in a little over a year.
I have similar conversations about Office 365. It costs twice as much as buying on-prem Microsoft Office, even if you buy Office every version. There's really no reason not to skip every other release of Office, which makes it practically four times the cost of on-prem.
If I have about 20 VMs running on a $10,000 server with a $6,000 Windows Server Datacenter license that has a five year warranty, let's math that out: The VMs cost $32,400. And I can spin up more VMs on that server at zero cost, it's not at capacity.
Obviously there's plenty of other little dollars and cents you can factor into an equation like this, but I'd argue Azure Reserved Pricing still has it's profit margin well defined.
I can definitely say last time I was in the job market, the majority of available IT jobs in my area was with MSPs, but there were tons of jobs available.
If you move to a cloud provider, change your processes, and reduce your number of IT people and use a combination of an MSP and a business support plan, you can save some money. If you only move to the cloud and don’t do either of the other two. You will spend a lot more money.
From the side of an IT person, it can go either way. Now that companies don’t need on prem IT people. The MSPs can rural source (get people a lot cheaper in lower cost of living areas) and outsource lower level IT folks. Either way there will be a lot of jobs, but there will be more people who can do it and wages will be lower for commodity IT folks.
On the other end, the true customer facing consultants can make a lot more. But if all you can do is click around on the AWS Console, don’t expect to make a lot.
I’m mostly a $long_time developer but I know my way around AWS and my next job will probably be a around AWS consultancy but I really want to stick to developing/architecting on top of AWS and not just doing “lift and shifts”.
MSPs can also standardized procedures across clients, automated processes at scale, etc.
If you’re just doing a lift and shift and porting a bunch of VMs it will cost more and unfortunately that’s all most outside AWS consultants know how to do. They were usually IT folks who managed some on prem resources, watched a few ACloudGuru videos, got one AWS cert and now market themselves as “Digital Transformation Consultants”, who can click around on the AWS website.
> "Be a developer, or get out of the tech industry."
I don't agree with this. There are plenty of jobs in operations that require the skills you have (e.g. standing up servers and automating the heck out of things)
Writing code and creating software was one of the few things that really clicked for me, so I can't share advice on that. One thing I can say is that all of the operations work is where I feel myself spinning and getting nowhere, I push through when I have to, but I don't always have hours to spend debugging an environment--i leave that to someone who can solve the same issue in minutes.
I'd guess at least over 50% of folks here would have gone through a similar phase at some point in their lives (learning block wise).
Based on my experience, especially over the past couple of years, I've realised that even if I can power through some of the subjects/topics which I find arcane (e.g. linear algebra, tensor calculus), I quickly lose what I've learnt if I don't get to apply the learning on a real project, not a toy one.
In your case, I think once you've familiarised yourself with the absolute basic syntax structure if python, go find couple of real world projects and just try to get them done, without worrying too much about code elegance. That would give you the necessary confidence and understanding for moving up the next step.
You can find small volunteer projects it participate in competitions (on platforms like kaggle) which may require you to use at least a portion of python.
But most importantly, don't lose heart. Learning takes time. It's just as important to take a step back from it for a while as it is to keep at it.
I strongly recommend a series of 4-5 video lectures on Coursera by a MIT prof I think. Course was called "Learning how to learn". You don't have to take everything in it but I still believe the overall approach may help you.
Cheers and all the best!
I think that some of my issues possibly stem from a combination of my previous job being a hostile work environment as well as repeatedly being told by other coworkers (and even a former manager) that: "The work that I was doing wasn't valuable."
I still haven't come to terms with it I think.
For what it's worth, I've been a developer professionally for 5 years and I still have this experience whenever I first look at code written by other people, no matter its quality.
It helps to keep your code so simple and peppered with comments than even an idiot (you) can understand it.
You don't learn to code by reading code, much like you don't learn to do math by reading equations, or carpentry by looking at houses. You have to get your hands dirty. Start with something "Hello World" level simple and add more bits as you go along. That's what we all do, really. The end result of such an iterative process will look inscrutable to others, but make perfect sense for you, because you made it.
Big dirty public secret: One guy's code being inscrutable to the next guy is why the next guy always argues for getting rid of the system that the previous guy developed. It's usually not because it's a bad system (your new system eventually becomes the next guy's "bad old" system, after all), but because no programmer thinks the same way.
Programming is very much a reflection of how a person thinks, of a mental model, and mental models are extremely individual things. Reading someone else's code is an exercise in empathy, and empathy isn't something programmers are known for. Programmers love to exist on their own little islands where everything works according to their internal logic. Teamwork means compromises. I once had to quit a job because I couldn't understand the internal logic of the team I had been hired to work with, as reflected in their codebase.
The main trait needed in a programmer is the ability to endure mind-bending levels of frustration. You have to keep hammering that nail until the problem is solved.
to which the answer is yes, ~95% of all developers endure struggles but fear that no one else does
Coding is difficult, don't let anyone tell you otherwise. It's completely normal to struggle. How long did it take you to learn to read and write English properly? 15-20 years?
Maybe try Kubernetes and container tech as a bridge before building actual apps?
Without perseverance and mental toughness it's going to be an uphill climb to become a programmer.
This is unfortunately how most of our organization works. For certain types of things, we're actually required to do this due to regulations, bureaucracy, and what not.
My advice would be to explore more areas of self improvement. It’s not only code out there!
You could look at continuous integration, focus on deployments, learn networking, go deep in cloud config management.
There’s also the people side, like project managing technical implementations, managing vendors, or doing budgets and planning.
You might also consider database admin which is really interesting work.
I guess what I’m saying is don’t feel limited by coding not being you thing. If you can learn enough to get by, you’ll find many roles where it’s not the core skill.
It does not have to be what you really want to learn, just one you can get stared with. Ruby has some decent lessons out there where they tell you what to type, you type it, and see the results.
First, like an essay, create an outline. Write what you want the output to look like, search stack overflow for examples, and it will get easier.
I learned Matlab, essentially deleted it from memory, and felt as you describe a few years later. I had to relearn it and just pushing myself with the pomodoro technique (20 min on, 5 min break) worked.
Good luck.
If you want to get into development, the next step might be to learn a new language. Ruby, Python, and Javascript are pretty popular scripting languages that will have some overlap with bash that you can connect to. Pick a language that works closest to how you think and work with it in place of what you would normally do with bash.
Most importantly, understand that you are in technical role but that role is not the summation of who you are.
Additionally, not all developers have sysadmin skills, so understand that what you provide is valuable. Don't be hard on yourself, you have your entire life to keep building on what you know and who you want to become.
If you stick with the traditional definition of SysAdmin, your job market becomes smaller every year. That doesn’t mean you won’t find a high paying job, it just means there are less opportunities and more competition. Are you good at it? Then perhaps you don’t need those DevOps skills. There will be enough opportunities for your lifetime. Then instead focus on branding yourself. Write online about your skills, share your knowledge on internet to create a strong and visible profile that you can reference to it. Don’t keep your knowledge on your head. Make it more visible even stuff that you think is obvious and easy. Then you always have a better chance in the market and it is very likely that good offers knock your door.
But, If you do feel that your career is at risk and you see kess chance in competing other SysAdmins, Then definitely invest on learning Cloud focused DevOps skills. And I think grasping the next step isn’t hard for a SysAdmin. Stuff like Modern CI/CD pipeline, Monitoring and integration with Cloud providers and different services, Cloud migration, some level of scripting probably in Python, etc.
And about coding, first of all you don’t have to become a programmer, . so be careful what resources and courses you take as some of them might be unnecessarily overwhelming. and secondly, like everything else, there is a learning curve. You just need to pass it and be patient.
I have been a sysadmin for 20 years and imagine I will be for another 20.
It runs on the same(ish) hardware you can find in your own server rack. It runs the same Linux (or some customly patched Linux derivative), it is connected by twisted pair or optics, it still uses the same tech you, a linux administrator, use underneath the whole shiny packaging. Kubernetes is a collection of scripts. Docker is just a script written in go to easier leverage linux kernel cgruops and namespaces.
To reiterate: cloud just hides this instead of replacing it completely. Worst case scenario, nobody (outside of amazon/google/azure/baidu) knows how to set up a simple webserver on a linux box in their closet - then your skills become invaluable for people not willing to keep their data/code/service god knows where with god knows who. There are still some people that value "on-premises/bare-metal" setup. So there's one venue. Another would be to deepen your sysadmin skills to the point where instead of working against the cloud, you could join them. As far as I know, they are constantly looking for good sysadmins.
Now if you really want to become a dev... Well, i would suggest probably to start with what you know best: bash scripts probably. Start there. Start by small chunks. Take a small script, try to understand how it works. Try to write a small script. Try to find tasks that would be useful / interesting for you, e.g. make a script that starts playing music in the morning, constantly and slowly increasing the volume until you wake up - just make stuff: the key here is to have fun while doing it and achieve the result, not make everything perfect.
And try to do this without any pressure on yourself. Sysadmins have value, and will continue to have value. You're doing this not because you're being obsoleted, but because you wish to better yourself.
Skip all the abstractions and push the numbers and currents.
Then when you get it, it is easier to go up in stack.
I wish I had found this book when I was starting out: https://www.htdp.org
I've only browsed it, but it looks like the best way to get started to me.
My brother worked through https://learncodethehardway.org python book, he liked that. I used the learn c the hard way and I like the guy's teaching style. He makes it clear that this stuff can be hard and you need to take your time.
I was constantly overwhelmed by programming, but I just kept going. I'm now at the point where I realise coding isn't the hardest part of the job anymore.
Are you thinking that everybody is migrating to cloud? For a business there is a lot of value in having their own local servers and infrastructure, even if small. Especially if they own data.
With that said, this industry is moving fast and if you wanna keep up you need to learn bash/python or powershell on windows, eventually... this is true. Or you can move to smaller companies with fewer assets to manage.
If you say you can throw a bash script together than that's a good start. When doing that, try to understand what the author has done - check how the script was made, look up each function and try to remember what they do.
Once you remember functions you can always use that knowledge and look up syntax and arguments in the docs and start writing your own code. Bash is not a good place to start though, because it has a lot of weird syntax but python is rather clean.
Also take your time, don't think it's something you can learn within a week or month, give it 6months - year, don't push it. If you can't figure it out today, it's ok, try another day.
I straddle both sides and I’d much rather deal with a seasoned software architect who comes to the cloud from that side than a sysadmin who comes from the other side.
A sysadmin doesn’t want to automate his job and let the cloud provider manage what they can manage. A good software Architect is mich more likely to be push as much off on to manage services as possible and automate as much as possible.
There may be some people like that, but I never said that I didn't want to automate my job. I'd rather automate as much as possible. I realize that in order to scale infrastructure, you can't be sitting around manually pressing buttons and running apt-get command to install software.
When the software architects come to you with a plan about how they plan to implement something, are you going you to understand the technology well enough to know that there is a managed alternative or will you just set up another VM?
I was a dev lead at my previous company and they decided to “move to the cloud”. They brought in a whole bunch of consultants. I didn’t know the first thing about AWS back then. I showed the consultants my architectural plans. All of this was greenfield development. If they had known anything about development and understood anything I was talking about, there is no way they wouldn’t have suggested that we used managed services in place of things I was proposing hosting in VMs.
Are you also willing to give up hosting things on VMs entirely and using managed services when possible. Will you even know how AWS’s services can be used in place of self hosted applications?
This isn’t aimed at you in particular. But there is a lot more to implementing systems on AWS cost efficiently than automating VMs.
Too many sysadmins turned “Cloud Architects” can’t get their mind around being “cloud native”.
I haven’t even mentioned Kubernetes or Docker - because I haven’t used either yet.
The consultants manually set up a VPC with different subnets in different availability zones just like they would have done on prem. No CloudFormation, no separate accounts under an organization, etc.
They couldn’t talk about using AWS as developers, or setting up a CI/CD pipeline using AWS’s services.
But I see your point, the problem with cloud services is that they don't share the same features so if you know AWS and then you move to azure you have to re-learn not just the naming but also how to work with a new toolset which is annoying, so most sysadmins will automate, but just not at the cloud provider level but below. I think this isn't wrong though because setting up an environment can be a lot more painful if you need to switch providers for whatever reason.
But I haven't worked in a large company, only below 200 employees so my thinking might be a little different.
I guess the larger issue is how much is the entire organization willing to rethink their architecture - hopefully led by the cloud experts - to be cloud native? If you are just hosting a bunch of VMs and not taking advantage of the providers manager services, from my experience, it will always cost more than just staying on prem.
Anyway I'm supposed to be a software developer and I end up spending a good chunk of time on sys admin stuff.
Yes, each of us.
Instead of being on premises, you should dive into cloud orchestration and management. You are still desperately needed as an IT person.
The analogy I like to use is: when building a house, you need a civil engineer and an architect. You wouldn't hire someone who does both right? Two very separate disciplines.
It's the same with programmers and IT. We both need each other. We write the code, you make sure that baby has enough room to grow and be secure. I would never attempt to set up AWS for anything serious because that's just asking to be hacked. I would recommend we hire a proper systems IT person.
I usually suggest tutorials, but I get the impression from them that the tutorials they use are long, boring, and worst of all.. too easy. Then they dive into real-world work and get overwhelmed. I feel like this is akin to showing them how interlocking gears work for the first time and then asking them to fix my car.
What helped me was being in that setting where I would get small not-real-world problems to solve over and over. IOW homework, classwork, etc.. I think is what helped. Eventually I did enough small things that when I saw real-world code, I could pick it apart into pieces that I understood because I've actually programmed those pieces before. Having written it before was a huge help as well because I already had to be familiar with the concepts to get it to work... reading it was then a matter of comparing what they have to what I wrote in the past and incrementing my knowledge (rather than starting from zero understanding).
TL;DR I think there is a gap between beginner/intermediate tutorials and real-world coding. I think you may need to find yourself a path through this gap that involves solving a lot of practice problems and working your way up.
HTH and Good Luck