Ask HN: Is my career doomed if I don’t become an AI/ML Developer?
A bit of background: I’ve been a software developer for about ten years now. I started doing JavaScript development and have transitioned in to more “back-end” development using Java, C#, and Python. However, I don’t have a computer science degree and my math education never progressed beyond trigonometry in college. That being said, I keep up on technology near every day and try to keep my skills as up-to-date as possible.
One of the major things I’ve seen all over HN and elsewhere is the emphasis on the growing AI/ML industry. I’ve tried to dig into both areas in some detail and find that the math is just way beyond me, leading me to believe that maybe it’s just not my cup of tea given that I don’t have a ton of extra time to teach myself calculus and advanced statistics. However, I’m concerned that if I don’t learn those things my career is going to be dead in ten years.
How realistic are these concerns? Should I be as worried as I am or will there be a need for good Java, etc., developers in the relatively near future?
72 comments
[ 3.2 ms ] story [ 113 ms ] thread2) In most ML projects, the majority of the necessary code written has nothing to do with ML, it's all the data transformation, transformation pipeline, service backend, data management tools, etc etc; so in a multi-person ML project usually most developers don't touch the ML part.
I am a statistician and a developer and my work falls into the domain of machine learning. The vast majority of my job is really painful data gathering, scrubbing, and fighting the toolchain. I have done this for YEARS, and the software is never just right, requires tons of tuning, huge budgets, and understanding management. These variables just don't line up that often. OP has nothing to worry about.
Moreover, the math stuff just gets abstracted away into a library. It's never like what went on in graduate school.
Try to make friends, get involved in projects.
Good luck!
Your question isn't to dissimilar from: Is my career as a backend developer doomed if I don't become a JavaScript/React developer?
1997: Is my career doomed if I don't become a web developer? No.
2007: Is my career doomed if I don't become a mobile developer? No.
2017: Is my career doomed if I don't become an AI/ML developer? No.
2027: Is my career doomed if I don't become a/an [...]? No.
If you're a programmer, focus on the fundamentals: Learn to write new code and maintain legacy code. Learn to write tests. Learn API design. Learn how to estimate projects. Learn to differentiate between a risky code change and one that isn't risky. Learn to refactor something without changing its functionality. Read/know the literature. Don't get hung up on any particular language, framework or technology.
I'm at a client site that is still rolling things from their MAINFRAMES into web apps. I know some old LISP programmers that are freaking out that major systems will die because the youngest person on the team is pushing 60 with no code or programmer replacement in site.
There's always a long tail, and always something new coming out.
Actually, be wary with this sort of "specialization" because if the market shrinks or stops growing, you'll end up isolated from the work force.
What you don't get from the flashy news articles is that a lot of ML work is programming and sys admin work: writing code to prepare the data, and a cycle of tweaking and re-running large jobs (including debugging them when they run out of resources, etc.).
You can also read papers like this which talk about the software engineering challenges of machine learning systems.
"Machine Learning: The High Interest Credit Card of Technical Debt"
https://research.google.com/pubs/pub43146.html
There's a big difference between "we got this thing to work once" (which is most of the new articles you read), and a system that does useful work over a long period of time. That latter is what you should know about as a software developer.
Basically, you have a training set of data that has "this is the data in" and "these are the outputs I expect." so if you are training something to learn the different animals of the animal kingdom, your data set might be portraits of giraffes and gorillas, and your outputs might be simple labels like "gorilla, snake, giraffe, tucan" etc. The neural net part of it is like the parameters you can hone. You can hone how many layers deep it is (try a layer for each dimension, like if you are tracking 5 facial features, you can try a network of roughly 5 layers). You have an "activation function" [usually sigmoid] because the _neurons_ sum their inputs and only fire on activation threshold (like your brain neurons).
Anyway, don't let the maths hold you back. When you can find a Python library on neural networks and play with it well enough to get results, you have pretty much figured out the puzzle. Not many people have deep intuition on how it works exactly, because the parameters (the "brain" of "neurons") is really an abstract mathematical object that is adjusted regressively over the course of learning [the training data].
Don't be discouraged, there is plenty of growth in software, and unless you are trying to help make a breakthrough in ML or "artifical intelligence" which, in my opinion, is something we have not even come close to touching, you don't really need to know the nitty gritty. The promising modern day results in ML came about because someone wanted to model a brain with a computer and ended up learning how to pattern match.
Remember "nanotechnology" from a few years ago? I recall reading, in a respected source, soon we would all have nanotechnology technology infused clothing off which would just roll all foreign particles! Well...still wearing my cotton slacks today...
If you want to get a look at how AI and ML are going to end up being applied to a huge number of business problems, try out the Azure Cognitive Services APIs[1]. I'm not associated with them in any way, I've just played around with the APIs and enjoyed them.
A huge number of developers today wouldn't be able to re-implement the frameworks they build applications on top of. And that doesn't stop them from being productive and well-paid. They add value be being able to to recognize real-world problems, and understand how various libraries and frameworks can be combined to solve those problems.
I think that AI/ML over the next 10 years will be like SaaS was over the previous 10 years. When frameworks like Rails became popular and gained wide acceptance, it became easy to very quickly come up with web-based solutions to a huge number of problems. And people didn't do this by being experts on how the whole tech stack worked. They did it by knowing how to solve problems with specific technologies, and then recognizing problems that the technology could solve. Lots of startups have made piles of money this way.
I'm a huge proponent of understanding as much as possible of the theory and math behind the technology to use. But being able to solve problems with the technology requires you to be out the world, recognizing problems that can be solved. For example: take a look at the kinds of output the Azure Computer Vision API can provide when provided with a given image[2]. Now think of all the businesses you've encountered during your career. Most of those businesses have problems you could solve using that API. And if you don't like MS or Azure, they're far from being the only provider of such APIs.
There is, and will continue to be, lots of opportunity (and money) available for people who can understand how to use new technology and recognize how to apply it to problems in a way that will save companies money, or help them make more money. Be one of those people.
[1]https://azure.microsoft.com/en-ca/services/cognitive-service... [2] https://azure.microsoft.com/en-ca/services/cognitive-service...
Analogously, my advanced calculus is pretty weak these days, but I have found a strong understanding of basic algebra and vector math is all you need to be a good graphics programmer. There are many fields like this where you do not need to read or understand a single whitepaper in order to be functional within it.
Old school ETL grew up along with the ML folks to be sophisticated, and essentially another tech. stack that looks like back end programming. Figuring out how to move data around reliably and quickly and clean it up in process is an unsexy job that's in really high demand. Using code (often JS) to pull out data and do mockups for the BI people before they plug in their fancy dataviz tools and do the pretty stuff is a lot like front end programming, and also not sexy, but in high demand.
But don’t say “I took Andrew Ng’s Coursera class and now I’m an expert on AI/ML!” like many engineers tend to do nowadays.
It reminds me of when I first started programming, I started seeing everything a little bit differently...I'm old and this will likely have little to no impact on my career but it's been a lot of fun.
I have no idea how to build an image processor, or a database, or cloud infrastructure, but I can use Pillow, Postgres, and AWS to build products on top of those capabilities. AI is starting down the same route, where the number of developers who use it will vastly outstrip the number of developers who contribute directly to it.
A lot of folks like to smugly predict the future based on their individual experience during past technology cycles. The advancements coming out from AI, however, are completely different than anything humans have seen before. It is entirely possible with deep learning that entire sectors of tech could be automated away. Maybe front end development doesn't get fully automated, but perhaps we build tech that makes a single developer 10-100x more productive? Then the labor demand for front end developers could drop in the same way that bank teller jobs have dropped due to ATMs and online banking. I would argue that while software is eating the world, AI is going to eat software.
Don't discount AI. Learn as much as you can about it and perhaps your career will be the last one to be automated away...
https://www.wsj.com/articles/technology-isnt-a-job-killer-14...
Jokes aside, you don't have to learn AI/ML specifically to remain relevant, but you have to learn something. You can learn:
- about a domain (energy, finance, medicine)
- a toolset (math, stats, programming, design),
- a "hot" new technology (bitcoin, drones, wearables, VR),
- put together a network of talented people and so on.
You have to do something, but not necessarily any particular thing.
The next part is controversial but I personally believe the first unimpressive but totally general AI systems will appear in 2019. They will be animal-like intelligences, with real or simulated embodiment, inputs and outputs that can serve any purpose, online learners that can reuse the same nets across tasks.
As AGI becomes more powerful in the early to mid 2020s it will become apparent that most of the existing jobs will either be replaced or have to change to something the AIs can't or aren't allowed to do. AIs will be able to code.