After seeing the fast progress of LLM, how long do you think it will take until writing code stops being necessary and SWEs shift to more business and architecture oriented roles?
I'm not sure, but to anybody who is interested in this question but hasn't spent much (or any) time using ChatGPT yet, or who hasn't already spent time thinking deeply about this, here's a video I'd suggest watching. It's Dave "EEVBlog" Jones, using ChatGPT to write the code for an Arduino project he was working on.
His goal was to find out "can ChatGTP generate all of the code I need for this project."
Spoilers: In the end he still had to make some minor modifications to the code by hand, but it came very close to generating the exact code he needed, although he did need a couple of rounds of going "back and forth" with ChatGPT to get it that close. Still, it was a moderately impressive demonstration of what ChatGPT can do today from a coding perspective.
Thanks for the video, I am definitive giving it a look. My question arouse from a similar experience.
I started to integrate ChatGPT and Codex into my workflow and I was surprised how useful their recommendations were for non trivial problems. Yes it required minor tweaks but if you know what you are doing it’s a huge productivity boost.
If it keeps improving at this rate I believe there is gonna be a point where actual coding stops being necessary.
> until writing code stops being necessary and SWEs shift to more business and architecture oriented roles
IC advancement ladder looks like that already. LLMs remove the justification for low effort, low quality developers, because they can simulate that and far better.
A long time. A lot of what I've read people are using it to create new software/code. The majority of time spent on an app running for 5-10 years isn't on writing new code. There are large areas of code just working perfectly that nobody has touched in years. Then a feature request comes in for that area and the majority of the time an engineer spends is reading and understanding the code and the new feature. It might just be a couple lines of code changes but several hours of discussion and understanding.
When it will be possible to feed the ai codebases of millions of lines and then feed it Jira tickets for new features and bug fixes, and then get the exact result you need.
That completely automates the whole loop. I am talking about picking Jira tickets and instead of coding the solution describe the different parts to an LLM and adapt it. In that case no code would be written just prompt or requirements engineering
One thing that is easily observed by the AI hype squad is that it's always a human that has to review the output since it is very clear that the AI output cannot be trusted to be correct as it outputs lots of incorrect bullshit.
So now whatever the output, it still requires a human who knows about 'coding' or some other expertise to review the generated output of the AI and they cannot assume that the answer given by the AI is even correct.
This hype is going to subside into lots of garbage, once all the low hanging fruit is picked and people realise that the outputs of LLMs like ChatGPT is untrustworthy.
Yes, but that does not mean it will write most code. I am not talking about programmers dissapearing, I am talking about them stopping to write React components.
Thing is, I don't technically "write" most of the code today. I type the first few characters and IntelliJ fills the rest for me (and also takes care of the imports and some other useful stuff). Is it literally one click or a prompt and all my code gets generated? Well no. But I know exactly what is going in my code without having to actively read it. In case some error slips in, I know what my intention was so corrections are trivial.
I dread reviewing someone else's code. I consider it the hardest part of my job as it requires a lot of concentration lest some bug creep in.
I am reminded of some Java codebases filled with unknown annotations whose annotation processors were in some library being imported. It was a nightmare trying to figure out what was happening and I dearly wanted to accost the engineer who wrote that mess. Unfortunately they had left the company.
This is like the "AI can't draw hands" argument. At the current rate of progress AI will be able to draw hands by next year. And it will be able to write increasingly correct code.
It wrote much worse code last year, and couldn't write passable code in 2021.
Not for a long time. I think LLMs can be as good as humans at coding and humans will still have coding jobs. Business people have to trust that the system being built is upto spec, that it will not break randomly because the machine took some instruction too literally, and more importantly, when shit hits the fan, they need actual bodies in the war room.
Will there be as many jobs as there are now? No idea but my guess is even if there is a shrinking of the market, AI will not be anywhere near the top cause. It will be more mundane reasons like the market itself stabilizing/shrinking, commodification of software skills etc. Said another way, the world does not need 1 million distributed systems experts. With or without LLMs, the bubble will pop someday and a lot of them are going to be out of work.
Program or be programmed. Write code to combat the adversity of programs which assault your everyday life. And 'AI' is just if...else statements, nothing more. Fight fire with fire.
Programming will always be important, it will never not be important. There will always be a point in time where someone needs to know how the sausage is made.
We still have people who need to know how the silicon functions, we still need people to synthesize the silicon primitives into something that can interpret assembly. We need people to write the translation layer between high level code and assembly. We need people working on the API that interacts with all the various hardware.
How do you think AI/ML systems will be troubleshot without people who understand how they are made?
Software is the process of creating and synthesizing abstractions. If you define programming as a specialty primarily concerned about the language/science/math of abstraction it will never go away. The layer of abstraction will change, but the labour of abstraction will always be there.
Even if you believe that AI will surpass us and will be able to invent and realize complex systems, that creates an existential threat to biological life, that type of battle would be fought with programmers. Even if you think we could program safety measures, gamma rays pelt the earth flipping bits on computer systems across the planet. Microsoft can detect solar activity by the number of crash reports they get. What happens when the wrong bit gets flipped? What happens when the wrong machine gets too hot?
I think the question you're asking might be closer to "What do you think the graph of aggregate %time spent on domain specific business logic over time looks like?"
I don't think writing software will be fundamentally different in my life time, faster as we codify more abstractions, but not different.
My guess is about 5 years till most startups use an LLM type copilot, 10 years for legacy companies, and then another 10 years after that for the ones that prefer older processes (think companies that avoided cloud etc).
When the internet became widely available you had a good long while where many people refused to use it because it was different from what they were used to. LLM's are already on the same arc, and you can already get internet like gains out of using them today.
Even after that I suspect a little code will be written by hand. The equivalent of writing assembly or C today. With how fast the LLM tech is moving though that may be naive.
18 comments
[ 3.6 ms ] story [ 61.6 ms ] threadHis goal was to find out "can ChatGTP generate all of the code I need for this project."
https://www.youtube.com/watch?v=g5_Ts9SWbYs
.
.
.
.
.
.
.
Spoilers: In the end he still had to make some minor modifications to the code by hand, but it came very close to generating the exact code he needed, although he did need a couple of rounds of going "back and forth" with ChatGPT to get it that close. Still, it was a moderately impressive demonstration of what ChatGPT can do today from a coding perspective.
I started to integrate ChatGPT and Codex into my workflow and I was surprised how useful their recommendations were for non trivial problems. Yes it required minor tweaks but if you know what you are doing it’s a huge productivity boost.
If it keeps improving at this rate I believe there is gonna be a point where actual coding stops being necessary.
IC advancement ladder looks like that already. LLMs remove the justification for low effort, low quality developers, because they can simulate that and far better.
Probably 50 years from now.
So now whatever the output, it still requires a human who knows about 'coding' or some other expertise to review the generated output of the AI and they cannot assume that the answer given by the AI is even correct.
This hype is going to subside into lots of garbage, once all the low hanging fruit is picked and people realise that the outputs of LLMs like ChatGPT is untrustworthy.
I dread reviewing someone else's code. I consider it the hardest part of my job as it requires a lot of concentration lest some bug creep in.
I am reminded of some Java codebases filled with unknown annotations whose annotation processors were in some library being imported. It was a nightmare trying to figure out what was happening and I dearly wanted to accost the engineer who wrote that mess. Unfortunately they had left the company.
It wrote much worse code last year, and couldn't write passable code in 2021.
Will there be as many jobs as there are now? No idea but my guess is even if there is a shrinking of the market, AI will not be anywhere near the top cause. It will be more mundane reasons like the market itself stabilizing/shrinking, commodification of software skills etc. Said another way, the world does not need 1 million distributed systems experts. With or without LLMs, the bubble will pop someday and a lot of them are going to be out of work.
We still have people who need to know how the silicon functions, we still need people to synthesize the silicon primitives into something that can interpret assembly. We need people to write the translation layer between high level code and assembly. We need people working on the API that interacts with all the various hardware.
How do you think AI/ML systems will be troubleshot without people who understand how they are made?
Software is the process of creating and synthesizing abstractions. If you define programming as a specialty primarily concerned about the language/science/math of abstraction it will never go away. The layer of abstraction will change, but the labour of abstraction will always be there.
Even if you believe that AI will surpass us and will be able to invent and realize complex systems, that creates an existential threat to biological life, that type of battle would be fought with programmers. Even if you think we could program safety measures, gamma rays pelt the earth flipping bits on computer systems across the planet. Microsoft can detect solar activity by the number of crash reports they get. What happens when the wrong bit gets flipped? What happens when the wrong machine gets too hot?
I think the question you're asking might be closer to "What do you think the graph of aggregate %time spent on domain specific business logic over time looks like?"
I don't think writing software will be fundamentally different in my life time, faster as we codify more abstractions, but not different.
When the internet became widely available you had a good long while where many people refused to use it because it was different from what they were used to. LLM's are already on the same arc, and you can already get internet like gains out of using them today.
Even after that I suspect a little code will be written by hand. The equivalent of writing assembly or C today. With how fast the LLM tech is moving though that may be naive.