True for now because models are mainly used to implement features / build small MVPs, which they’re quite good at.
The next step would be to have a model running continuously on a project with inputs from monitoring services, test coverage, product analytics, etc. Such an agent, powered by a sufficient model, could be considered an effective software engineer.
We’re not there today, but it doesn’t seem that far off.
This is a good headline. LLMs are remarkably good at writing code. Writing code isn't the same thing as delivering working software.
A human expert needs to identify the need for software, decide what the software should do, figure out what's feasible to deliver, build the first version (AI can help a bunch here), evaluate what they've built, show it to users, talk to them about whether it's fit for purpose, iterate based on their feedback, deploy and communicate the value of the software, and manage its existence and continued evolution in the future.
Some of that stuff can be handled by non-developer humans working with LLMs, but a human expert needs who understands code will be able to do this stuff a whole lot more effectively.
I guess the big question is if experienced product management types can pick up enough coding technical literacy to work like this without programmers, or if programmers can pick up enough enough PM skills to work without PMs.
My money is on both roles continuing to exist and benefit from each other, in a partnership that produces results a lot faster because the previously slow "writing the code" part is a lot faster than it used to be.
what if the real paradigm shift isn't about replacing engineers in building durable systems, but about making software so cheap and disposable that the concept of "technical debt" becomes irrelevant for a new class of "single-use" or ultra-short-lifespan applications?
The context windows are still dramatically too small and the models aren’t yet seeming to train on how to build maintainable software. There is a lot less written down about how to do this on the public web. There’s a bunch of high level public writing but not may great examples of real world situations that happen on every proprietary software project, because that’s very messy data locked away internal to companies.
I’m sure it’ll improve over time but it won’t be nearly as easy as making ai good at coding.
Yes, they're bad now, but they'll get better in a year.
If the generative ability is good enough for small snippets of code, it's good enough for larger software that's better organized. Maybe the models don't have enough of the right kind of training data, or the agents don't have the right reasoning algorithms. But it is there.
I see so many people on the Internet who claim they can fix AI VIBE Code. Nothing new I've been Super Debugging crappy code for 30 years to make it work.
I've been forcing myself to "pure vibe-code" on a few projects, where I don't read a single line of code (even the diffs in codex/claude code).
Candidly, it's awful. There are countless situations where it would be faster for me to edit the file directly (CSS, I'm looking at you!).
With that said, I've been surprised at how far the coding agents are able to go[0], and a lot less surprised about where I need to step in.
Things that seem to help:
1. Always create a plan/debug markdown file
2. Prompt the agent to ask questions/present multiple solutions
3. Use git more than normal (squash ugly commits on merge)
Planning is key to avoid half-brained solutions, but having "specs" for debug is almost more important. The LLM will happily dive down a path of editing as few files as possible to fix the bug/error/etc. This, unchecked, can often lead to very messy code.
Prompting the agent to ask questions/present multiple solutions allows me to stay "in control" over the how something is built.
I now basically commit every time a plan or debug step is complete. I've tried having the LLM control git, but I feel that it eats into the context a bit too much. Ideally a 3rd party "agent" would handle this.
The last thing I'll mention is that Claude Code (Sonnet 4.5) is still very token-happy, in that it eagerly goes above and beyond when not always necessary. Codex (gpt-5-codex) on the other hand, does exactly what you ask, almost to a fault. For both cases, this is where planning up-front is super useful.
[0]Caveat: the projects are either Typescript web apps or Rust utilities, can't speak to performance on other languages/domains.
>> Codex (gpt-5-codex) on the other hand, does exactly what you ask, almost to a fault.
I've seriously tried gpt-5-codex at least two dozen times since it came out, and every single time it was either insufficient or made huge mistakes. Even with the "have another agent write the specs and then give it to codex to implement" approach, it's just not very good. It also stops after trying one thing and then says "I've tried X, tests still failing, next I will try Y" and it's just super annoying. Claude is really good at iterating until it solves the issue.
Sonnet 4.5 is rebranded Opus 4. That's where it got its token-happiness.
Try asking Opus to generate a simple application and it'll do it. It'll also add thousands of lines of setup scripts and migration systems and Dockerfiles and reports about how it built everything and... Ooof.
Sonnet 4.5 is the same, but at a slightly smaller scale. It still LOVES to generate markdown reports of features it did. No clue why, but by default it's on, you need to specifically tell it to stop doing that.
Software engineering has always been about managing complexity, not writing code. Code is just the artifact. No-code, low-code is all code but doesn't make for a good software engineered application
The problem with vibe coding is it demoralizes experienced software engineers. I'm developing a MVP with vibes and Claude Code and Codex output work in many cases for this relatively new project. But the quality of code is bad. There is already duplicated or unused logic, a lot of code is unnecessarily complex (especially React and JSX). And there's little PR reviews so that "we can keep velocity". I'm paying much less attention for quality now. After all, why bother when AI produce working code? I can't justify and don't have energy for deep-diving system design or dozens of nitpicking change requests. And it makes me more and more replaceable by LLM.
I've been working with a data processing pipeline that was vibe-coded by an AI engineer, and while the code works, as software that has to fit into a production environment, it's a mess. Take logging for example. The pipeline is made up of AWS lambdas written in python. The person who built it wanted to add context to each log for debugging and the LLM generated hundreds of lines of python in each lambda to do this (no common library). But he (and the LLM) didn't understand that there were a bunch of files that initialized their own loggers at the top of the file, so all that code to set context in the root logger wouldn't get used in those files. And then he wanted to parallelize some tasks, and both he and the LLM didn't understand that the logging context was thread-local and wouldn't show up in logs generated in another thread. So what we ended up with was 250+ line logging_config.py files in each individual lambda that were only used for a small portion of the logs generated by the application.
> I don’t really know why AI can't build software (for now)
Could be because programming involves:
1. Long chains of logical reasoning, and
2. Applying abstract principles in practice (in this case, "best practices" of software engineering).
I think LLMs are currently bad at both of these things. They may well be among the things LLMs are worst at atm.
Also, there should be a big asterisk next to "can write code". LLMs do often produce correct code of some size and of certain kinds, but they can also fail at that too frequently.
I've been experimenting with a little vibe coding.
I've generally found the quality of .NET to be quite good. It trips up sometimes when linters ping it for rules not normally enforced, but it does the job reasonably well.
The front-end javascript though? It's both an absolute genuis and a complete menace at the same time. It'll write reams of code to gets things just right but with no regards to human maintainability.
I lost an entire session to the fact that it cheerfully did:
npm install fabric
npm install -D @types/fabric
Now that might look fine, but a human would have realised that the typings library is a completely different out-dated API, the package last updated 6 years ago.
Claude however didn't realise this, and wrote a ton of code that would pass unit tests but fail the type check. It'd check the type checker, re-write it all to pass the type checker, only for it now to fail the unit tests.
Eventually it semi-gave up typing and did loads of (fabric as any) all over the place, so now it just gave runtime exceptions instead.
I intervened when I realised what it was doing, and found the root cause of it's problems.
It was a complete blindspot because it just trusted both the library and the typechecker.
So yeah, if you want to snipe a vibe coder, suggest installing fabricjs with typings!
Instead of just committing more often, make the agent write commits following the conventional commits spec (feat:, fix:, refactor:) and reference a specific item from your plan.md in the commit body. That way you’ll get a self-documenting history - not just of the code, but of the agent’s thought process, which is priceless for debugging and refactoring later on
I think this can be extended (but not necessarily fully mitigated) by working with non-SWE agents interacting with the same codebase. Drafting product requirements, assess business opportunities, etc. can be done by LLMs.
I feel you can apply this to all roles. When models passed highschool exam benchmarks, some people talked as if that made the model equivalent to a person passing highschool. I may be wrong, but I bet even an state of the art LLM couldn't complete high school. You have to do things like attending classes at the right time/place, take initiative, keep track of different classes. All of the bigger picture thinking and soft skills that aren't in a pure exam.
Improving this is what everyone's looking into now. Even larger models, context windows, adding reasoning, or something else might improve this one day.
And here I am, using AI twice within the last 12 hours, to ask it two questions about an extremely well used, extremely well documented, physics library, and both times having it return to me sample code which makes use of library methods which don't exist. When I tell it this, I get the "Oh, you're so right to point that out!" response, and get new code returned, which still just blatantly doesn't work.
Someone had a blog post that said if a LLM hallucinates a method in your library, that means it should statistically have a method like that. LLMs work on probabilities and if the math says something should be there, who are you to argue =)
Also use MCPs like codex7 and Agentic LLMs for more interactivity instead of just relying on a raw model.
Even the code quality is often quite poor. At the same time, not using critical thinking can have serious consequences for those who treat AI as more than an explorer or companion. You might think that with AI, the number of highly skilled developers would increase but it could be quite the opposite. Code is just a medium; developers are paid to solve problems, not to write code. But writing code is still important as it refines your thoughts and sharpens your problem-solving skills.
The human brain learns through mistakes, repetition, breaking down complex problems into simpler parts, and reimagining ideas. The hippocampus naturally discards memories that aren’t strongly reinforced.. so if you rely solely on AI, you’re simply not going to remember much.
I'm of the opinion that not a single software engineer has yet lost their job to AI.
Any company claiming they've replaced engineers with AI has done so in an attempt to cover up the real reasons they've gotten rid of a few engineers. "AI automating our work" sounds much better to investors than "We overhired and have to downsize".
I definitely disagree. I'm a software engineer, but have been heavily using AI the last few months and have gotten multiple apps to production since then. I have to guide the LLM along, yes, but it's perfectly capable of doing everything needed up to and including building the cloudformation templates for Fargate or whatever.
In some ways, this seems backwards. Once you have a demo that does the right thing, you have a spec, of sorts, for what's supposed to happen.
Automated tooling that takes you from demo to production ready ought to be possible. That's a well-understood task. In restricted domains, such as CRUD apps, it might be automated without "AI".
I've worked in a few teams where some member of the [human] team could be described as "Joe can code, but he can't build software."
The difference is what we used to call the "ilities": Reliability, inhabitability, understandability, maintainability, securability, scalability, etc.
None of these things are about the primary function of the code, i.e. "it seems to work." In coding, "it seems to work" is good enough. In software engineering, it isn't.
48 comments
[ 3.4 ms ] story [ 81.9 ms ] threadThe next step would be to have a model running continuously on a project with inputs from monitoring services, test coverage, product analytics, etc. Such an agent, powered by a sufficient model, could be considered an effective software engineer.
We’re not there today, but it doesn’t seem that far off.
A human expert needs to identify the need for software, decide what the software should do, figure out what's feasible to deliver, build the first version (AI can help a bunch here), evaluate what they've built, show it to users, talk to them about whether it's fit for purpose, iterate based on their feedback, deploy and communicate the value of the software, and manage its existence and continued evolution in the future.
Some of that stuff can be handled by non-developer humans working with LLMs, but a human expert needs who understands code will be able to do this stuff a whole lot more effectively.
I guess the big question is if experienced product management types can pick up enough coding technical literacy to work like this without programmers, or if programmers can pick up enough enough PM skills to work without PMs.
My money is on both roles continuing to exist and benefit from each other, in a partnership that produces results a lot faster because the previously slow "writing the code" part is a lot faster than it used to be.
I’m sure it’ll improve over time but it won’t be nearly as easy as making ai good at coding.
Yes, they're bad now, but they'll get better in a year.
If the generative ability is good enough for small snippets of code, it's good enough for larger software that's better organized. Maybe the models don't have enough of the right kind of training data, or the agents don't have the right reasoning algorithms. But it is there.
Candidly, it's awful. There are countless situations where it would be faster for me to edit the file directly (CSS, I'm looking at you!).
With that said, I've been surprised at how far the coding agents are able to go[0], and a lot less surprised about where I need to step in.
Things that seem to help: 1. Always create a plan/debug markdown file 2. Prompt the agent to ask questions/present multiple solutions 3. Use git more than normal (squash ugly commits on merge)
Planning is key to avoid half-brained solutions, but having "specs" for debug is almost more important. The LLM will happily dive down a path of editing as few files as possible to fix the bug/error/etc. This, unchecked, can often lead to very messy code.
Prompting the agent to ask questions/present multiple solutions allows me to stay "in control" over the how something is built.
I now basically commit every time a plan or debug step is complete. I've tried having the LLM control git, but I feel that it eats into the context a bit too much. Ideally a 3rd party "agent" would handle this.
The last thing I'll mention is that Claude Code (Sonnet 4.5) is still very token-happy, in that it eagerly goes above and beyond when not always necessary. Codex (gpt-5-codex) on the other hand, does exactly what you ask, almost to a fault. For both cases, this is where planning up-front is super useful.
[0]Caveat: the projects are either Typescript web apps or Rust utilities, can't speak to performance on other languages/domains.
I've seriously tried gpt-5-codex at least two dozen times since it came out, and every single time it was either insufficient or made huge mistakes. Even with the "have another agent write the specs and then give it to codex to implement" approach, it's just not very good. It also stops after trying one thing and then says "I've tried X, tests still failing, next I will try Y" and it's just super annoying. Claude is really good at iterating until it solves the issue.
Try asking Opus to generate a simple application and it'll do it. It'll also add thousands of lines of setup scripts and migration systems and Dockerfiles and reports about how it built everything and... Ooof.
Sonnet 4.5 is the same, but at a slightly smaller scale. It still LOVES to generate markdown reports of features it did. No clue why, but by default it's on, you need to specifically tell it to stop doing that.
Could be because programming involves:
1. Long chains of logical reasoning, and
2. Applying abstract principles in practice (in this case, "best practices" of software engineering).
I think LLMs are currently bad at both of these things. They may well be among the things LLMs are worst at atm.
Also, there should be a big asterisk next to "can write code". LLMs do often produce correct code of some size and of certain kinds, but they can also fail at that too frequently.
I've generally found the quality of .NET to be quite good. It trips up sometimes when linters ping it for rules not normally enforced, but it does the job reasonably well.
The front-end javascript though? It's both an absolute genuis and a complete menace at the same time. It'll write reams of code to gets things just right but with no regards to human maintainability.
I lost an entire session to the fact that it cheerfully did:
Now that might look fine, but a human would have realised that the typings library is a completely different out-dated API, the package last updated 6 years ago.Claude however didn't realise this, and wrote a ton of code that would pass unit tests but fail the type check. It'd check the type checker, re-write it all to pass the type checker, only for it now to fail the unit tests.
Eventually it semi-gave up typing and did loads of (fabric as any) all over the place, so now it just gave runtime exceptions instead.
I intervened when I realised what it was doing, and found the root cause of it's problems.
It was a complete blindspot because it just trusted both the library and the typechecker.
So yeah, if you want to snipe a vibe coder, suggest installing fabricjs with typings!
Instead of just committing more often, make the agent write commits following the conventional commits spec (feat:, fix:, refactor:) and reference a specific item from your plan.md in the commit body. That way you’ll get a self-documenting history - not just of the code, but of the agent’s thought process, which is priceless for debugging and refactoring later on
Improving this is what everyone's looking into now. Even larger models, context windows, adding reasoning, or something else might improve this one day.
Also use MCPs like codex7 and Agentic LLMs for more interactivity instead of just relying on a raw model.
The human brain learns through mistakes, repetition, breaking down complex problems into simpler parts, and reimagining ideas. The hippocampus naturally discards memories that aren’t strongly reinforced.. so if you rely solely on AI, you’re simply not going to remember much.
Any company claiming they've replaced engineers with AI has done so in an attempt to cover up the real reasons they've gotten rid of a few engineers. "AI automating our work" sounds much better to investors than "We overhired and have to downsize".
In some ways, this seems backwards. Once you have a demo that does the right thing, you have a spec, of sorts, for what's supposed to happen. Automated tooling that takes you from demo to production ready ought to be possible. That's a well-understood task. In restricted domains, such as CRUD apps, it might be automated without "AI".
To really get the most out of it though, you still need to have solid knowledge in your own field.
The difference is what we used to call the "ilities": Reliability, inhabitability, understandability, maintainability, securability, scalability, etc.
None of these things are about the primary function of the code, i.e. "it seems to work." In coding, "it seems to work" is good enough. In software engineering, it isn't.