First, skilled engineers using LLMs to code also think and discuss and stare off into space before the source code starts getting laid down. In fact: I do a lot, lot more thinking and balancing different designs and getting a macro sense of where I'm going, because that's usually what it takes to get an LLM agent to build something decent. But now that pondering and planning gets recorded and distilled into a design document, something I definitely didn't have the discipline to deliver dependably before LLM agents.
Most of my initial prompts to agents start with "DO NOT WRITE ANY CODE YET."
Second, this idea that LLMs are like junior developers that can't learn anything. First, no they're not. Early-career developers are human beings. LLMs are tools. But the more general argument here is that there's compounding value to working with an early-career developer and there isn't with an LLM. That seems false: the LLM may not be learning anything, but I am. I use these tools much more effectively now than I did 3 months ago. I think we're in the very early stages of figuring how to get good product out of them. That's obvious compounding value.
I think this would benefit from examples of including coding assistants in the stages enumerated; how can the agent be included in each stage? I've seen posts about successful collaboration with agents at say Google, where there is tons of upfront work among humans to agree on design, then work with the agent to build out parts of the project and ensuring thorough test suites are included.
Does including an agent at each stage of this cycle mean "context engineering"? Is this then just more text and assets to feed in at each stage of LLM ussage to provide the context for the next set of tokens to generate for the next stage of the cycle? Is there something deeper that can be done to encode this level of staged development into the agent's weights/"understanding"? Is there an established process for this yet?
> While the LLMs get to blast through all the fun, easy work at lightning speed, we are then left with all the thankless tasks: testing to ensure existing functionality isn’t broken, clearing out duplicated code, writing documentation, handling deployment and infrastructure, etc.
I’ve found LLMs just as useful for the "thankless" layers (e.g. tests, docs, deployment).
The real failure mode is letting AI flood the repo with half-baked abstractions without a playbook. It's helpful to have the model review the existing code and plan out the approach before writing any new code.
The leverage may be in using LLMs more systematically across the lifecycle, including the grunt work the author says remains human-only.
> lack in-depth knowledge of your business, codebase, or roadmap
So give them some context. I like Cline's memory bank approach https://docs.cline.bot/prompting/cline-memory-bank which includes the architecture, progress, road map etc. Some of my more complex projects use 30k tokens just on this, with the memory bank built from existing docs and stuff I told the model along the way. Too much context can make models worse but overall it's a fair tradeoff - it maintains my coding style and architecture decisions pretty well.
I also recommend in each session using Plan mode to get to a design you are happy with before generating any code.
one axis that is missing from the discussion is how fast they are improving. We need ~35 years to get a senior software engineer (from birth to education to experience). These things are not even 3.5 years old. I am very interested in this space, if you are too dm me on X:@fabmilo I am in SF.
I would love to see an anti-AI take that doesn't hinge on the idea that technology forces people to be lazy/careless/thoughtless.
The plan-build-test-reflect loop is equally important when using an LLM to generate code, as anyone who's seriously used the tech knows: if you yolo your way through a build without thought, it will collapse in on itself quickly. But if you DO apply that loop, you get to spend much more time on the part I personally enjoy, architecting the build and testing the resultant experience.
> While the LLMs get to blast through all the fun, easy work at lightning speed, we are then left with all the thankless tasks
This is, to me, the root of one disagreement I see playing out in every industry where AI has achieved any level of mastery. There's a divide between people who enjoy the physical experience of the work and people who enjoy the mental experience of the work. If the thinking bit is your favorite part, AI allows you to spend nearly all of your time there if you wish, from concept through troubleshooting. But if you like the doing, the typing, fiddling with knobs and configs, etc etc, all AI does is take the good part away.
For me it's simply this: the best thing about computers and programming is that they do exactly what the code I write says they'll do. That is a quality that humans and human/natural languages don't have. To me, LLMs feel like replacing the best property of computers with a (in this context) terrible property of humans.
Why would I want a fuzzy, vague, imprecise, up-to-interpretation programming language? I already have to struggle with that in documentation, specifications, peers and – of course – myself. Why would I take the one precise component and make it suffer from the same?
This contrasts of course with tasks such as search, where I'm not quite able to precisely express what I want. Here I find LLMs to be a fantastic advance. Same for e.g. operations between imprecise domains, like between natural languages.
> There's a divide between people who enjoy the physical experience of the work and people who enjoy the mental experience of the work.
Does this divide between "physical" and "mental" exist? Programming languages are formal languages that allow you to precisely and unambiguously express your ideas. I would say that "fiddling" with the code (as you say) is a kind of mental activity.
If there is actually someone out there that only dislikes AI coding assistants because they enjoy the physical act of typing and now have to do less of it (I have not seen this blog post yet), then I might understand your point.
> There's a divide between people who enjoy the physical experience of the work and people who enjoy the mental experience of the work
Eh, physical and mental isn't the divide — it's more like people who enjoy code itself as a craft and people who simply see it as a means to an end (the application). Much like a writer might labor over their prose (the code) while telling a story (the application). Writing code is far more than the physical act of typing to those people.
> The plan-build-test-reflect loop is equally important when using an LLM to generate code, as anyone who's seriously used the tech knows
Yeah I'm actually quite surprised that so many people are just telling AI to do X without actually developing a maintainable plan to do so first. It's no wonder that so many people are anti-vibe-coding — it's because their exposure to vibe coding is just telling Replit or Claude Code to do X.
I still do most of my development in my head, but I have a go-to prompt I ask Claude code when I'm stuck: "without writing any code, and maintaining existing patterns, tell me how to do X." it'd spit out some stuff, I'd converse with it to make sure it is a feasible solution that would work long term, then I tell it to execute the plan. But the process still starts in my head, not with a prompt.
> Test-Driven Development: generating extensive test cases prior to implementation to guide implementation and prevent regression.
I’ve found this concept trips CC up—- assertions are backwards, confusing comments in the test, etc. Just starting a prompt with “Use TDD to…” really helps.
This article is a great example of how the human is struggling to extrapolate what happens next. There won't be any humans anywhere near this part of the tech stack, just like no one building a SAAS writes assembly code; or has to put together their own server cluster a datacenter (remember pre-cloud?) for their company anymore. He's dead Jim. No one is telling anyone to ship the ML code without any testing. Human coders also make mistakes. I'll bet that in a few years product managers / QA people would rather work with an ML stack to generate code than a human engineering team. It'll not just be cheaper and faster, but a lot less hassle & more accurate. As an example, Python has roughly ~100 or so "keywords", and extensive public libraries and open source algorithms to call upon. Anyone who thinks this presents any sort of challenge for an LLM to profoundly master is is delusional. They can do IMO-grade math, and help proving novel theorems. They can code your YC startup just fine.
I do appreciate that this article moves past absolute negativity on LLMs and actually speaks to the fact that they are extremely useful for well defined programming tasks. I'm a bit sick of articles that are just pure negativity on these tools.
I will raise that LLMs are pretty good at some of the non-coding tasks too.
eg. "I'm currently creating an AI for a turn based board game. Without doing any implementation, create a plan for the steps that need to be done including training off real world game data".
The LLM creates a tasklist for iterative steps to accomplish the above. It usually needs correction specific to the business/game needs but it's a great start and i recommend doing this just so the LLM has a doc with context on what its trying to achieve in a bigger picture as you have it complete tasks.
LLM coding agents can't learn from experience on our code, but we can learn from using them on our code, and in the context of our team and processes. I started creating some harnesses to help get more of what we want from these tools, and less of what we need to work too much on - eg, creating specialized agents to refactor code and test after it's been generated, and make it more in line with our standards, removing bogus tests, etc. The learning is embedded in the prompts for these agents.
I think that this approach can already get us pretty far. One thing I'm missing is tooling to make it easier to build automation on top of, eg, Claude Code, but I'm sure it's going to come (and I'm tempted to try vibe coding it; if only I had the time).
IMO a vibe coder who is speaking their ideas to an agent which implements them is going to have way more time to think than a hand coder who is spending 80% of their time editing text.
Especially with junior engineers it is helpful to ask them to provide a Loom video or other proof that they have verified a feature or bug works as intended. I have tried setting up Claude Code with playwright to verify it's work, but so far am not very satisfied with the results. Any tools that are helpful with this end to end testing for web apps using Claude Code and other AI assistants? Feel free to share your product if it is relevant.
Not in my experience. I still spend much of the time thinking before prompting. Then I spend time reviewing the AI written code before using it. Does not feel like a trap. It mostly feels like having a super experienced pair programmer. I may be using it differently than others since I do not have it integrated to my IDE. I use it like I used google + stackoverflow before it.
the flaw in the article is acting like engineers always have a choice. the writers presents the contrasts of "fair delegation vs mollycoddling" mirroring "ai-driven development vs vibe coding"... but that sacrifice for short-term gain at the expense of scale is often draconically enforced.
obviously good and experienced engineers aren't going to be vibe coders/mollycoddlers by nature. but many good and experienced engineers will be pressured to make poor decisions by impatient business leaders. and that's the root of most AI anxiety: we all know it's going to be used as irresponsibly and recklessly as possible. it's not about the tech. it's about a system with broken incentives.
Has anyone read up on the recent paper from Meta/FAIR -- CWM: An Open-Weights LLM for Research on Code Generation with World Models
Which looks to attempt to give better "coding" understanding to the model instead of mere tokens and positioning and hence improve the coding capabilities of these "brilliant but unpredictable junior engineer" coding agents:
If you know, understand that you are in possession of a massive and temporary information asymmetry advantage, and you should run with it as hard and fast as you can to gain the biggest lead possible, before the author and the rest of the world gain that advantage too. Go, go now, go fast, do it in parallel, and don’t stop until you win. Opportunities like this are extremely rare not just in your life, but in the history of our society. Best of luck.
Broadly the critique is valid where it applies; I don’t know if it accurately captures the way most people are using LLMs to code, so I don’t know that it applies in most case.
My one concrete pushback to the article is that it states the inevitable end result of vibe coding is a messy unmaintainable codebase. This is empirically not true. At this point I have many vibecoded projects that are quite complex but work perfectly. Most of these are for my private use but two of them serve in a live production context. It goes without saying that not only do these projects work, but they were accomplished 100x faster than I could have done by hand.
Do I also have vibecoded projects that went of the rails? Of course. I had to build those to learn where the edges of the model’s capabilities are, and what its failure modes are, so I can compensate. Vibecoding a good codebase is a skill. I know how to vibecode a good, maintainable codebase. Perhaps this violates your definition of vibecoding; my definition is that I almost never need to actually look at the code. I am just serving as a very hands-on manager. (Though I can look at the code if I need to - have 20 years of coding experience. But if I find that I need to look at the code, something has already gone badly wrong.)
Relevant anecdote: A couple of years ago I had a friend who was incredibly skilled at getting image models to do things that serious people asserted image models definitely couldn’t do at the time. At that time there were no image models that could get consistent text to appear in the image, but my friend could always get exactly the text you wanted. His prompts were themselves incredible works of art and engineering, directly grabbing hold of the fundamental control knobs of the model that most users are fumbling at.
Here’s the thing: any one of us can now make an image that is better than anything he was making at the time. Better compositionality, better understanding of intent, better text accuracy. We do this out of the box and without any attention paid to promoting voodoo at all. The models simply got that much better.
In a year or two, my carefully cultivated expertise around vibecoding will be irrelevant. You will get results like mine by just telling the model what you want. I assert this with high confidence. This is not disappointing to me, because I will be taking full advantage of the bleeding edge of capabilities throughout that period of time. Much like my friend, I don’t want to be good at managing AIs, I want to realize my vision.
For now, my mind is still made up. I leave the door open to be shown any serious piece of software that is built primarily through agentic workflows. Having tried to use these tools over the past month to build a critical piece of infrastructure for my company, I agree with OP. I spent so much time wrangling back unnecessary garbage that the LLM found was important, that I wondered if just writing it in one shot would have been actually faster. Simple things like 'test all this workflow logic' resulted in the LLM inserting a non-sensical mock at the top of the test file that took me an hour or two to unwind.
Other than that, I keep hearing the same arguments - "LLMs free up more time for me to think about the 'important things'." Son, your system is not durable, your tests are misleading, and you can't reason about what's happening because you didn't write it. What important things are left to think about??
I cannot express how tired I am of seeing this beyond stupid take.
If you truly believe that, you have either only ever worked with the most piss-poor junior engineers, or you simply have never worked with junior engineers.
LLMs do not learn, LLMs do not ask clarifications, LLMs do not wonder if they're going in the wrong direction, LLMs do not have taste, LLMs do not have opinions, LLMs write absolutely illogical nonsense, LLMs do not ask themselves what is the best for the customer, LLMs have no context beyond what you have explicitely fed them for that one specific task, and much more.
suggests a workflow where AI is used almost exclusively to speed up the writing of known, targeted code whose structure has already been thought out. And possibly as a (non-coding) sounding board during the thinking out.
Even outside of AI coding I have found a tremendous amount of value in using AI to produce a requirements and spec document for me to code from. The key unlock for me is asking AI to “interview” me about how this system/feature should work. As part of that process it will often ask a question that gets me thinking about interesting edge cases.
I will say I always provide an initial context document about the feature/system, to avoid us starting with trivial questions. After about 45minutes I’ll often feel I’ve covered enough ground and given the problem enough thought to really put pen to paper. Off the back of this I’ll ask it to summarise the spec and produce a document. This can be a good point to ditch AI if you are so inclined but still get value from it.
Effective AI coding is actually extremely slow if you take into account an exhaustive planning stage where the task specification is laid down in sufficient and unambiguous detail. I had to get the LLM to review my spec over twenty times, always freshly, before I thought it was good enough to be implemented well. Also, it really helps for multiple diverse LLMs to review the spec, as they all have their unique insights. In this way, AI coding also helps me avoid numerous bugs that could have left me in trouble if not for the AI.
Once the planning is done, the actual coding is very fast. The human review that follows is again slow, often also leading to minor new tickets.
I think the post, while extensive, missed one important issue.
The fact then when we read others' code, we don't remember/integrate it into our thinking as well as we do when we're the authors. So mentoring "AI Juniors" provides less growth then doing the job, esp. if it is mostly corrective actions.
79 comments
[ 3.1 ms ] story [ 68.9 ms ] threadFirst, skilled engineers using LLMs to code also think and discuss and stare off into space before the source code starts getting laid down. In fact: I do a lot, lot more thinking and balancing different designs and getting a macro sense of where I'm going, because that's usually what it takes to get an LLM agent to build something decent. But now that pondering and planning gets recorded and distilled into a design document, something I definitely didn't have the discipline to deliver dependably before LLM agents.
Most of my initial prompts to agents start with "DO NOT WRITE ANY CODE YET."
Second, this idea that LLMs are like junior developers that can't learn anything. First, no they're not. Early-career developers are human beings. LLMs are tools. But the more general argument here is that there's compounding value to working with an early-career developer and there isn't with an LLM. That seems false: the LLM may not be learning anything, but I am. I use these tools much more effectively now than I did 3 months ago. I think we're in the very early stages of figuring how to get good product out of them. That's obvious compounding value.
Does including an agent at each stage of this cycle mean "context engineering"? Is this then just more text and assets to feed in at each stage of LLM ussage to provide the context for the next set of tokens to generate for the next stage of the cycle? Is there something deeper that can be done to encode this level of staged development into the agent's weights/"understanding"? Is there an established process for this yet?
- Specification
- Documentation
- Modular Design
- Test-Driven Development
- Coding Standard
- Monitoring & Introspection
I’ve found LLMs just as useful for the "thankless" layers (e.g. tests, docs, deployment).
The real failure mode is letting AI flood the repo with half-baked abstractions without a playbook. It's helpful to have the model review the existing code and plan out the approach before writing any new code.
The leverage may be in using LLMs more systematically across the lifecycle, including the grunt work the author says remains human-only.
So give them some context. I like Cline's memory bank approach https://docs.cline.bot/prompting/cline-memory-bank which includes the architecture, progress, road map etc. Some of my more complex projects use 30k tokens just on this, with the memory bank built from existing docs and stuff I told the model along the way. Too much context can make models worse but overall it's a fair tradeoff - it maintains my coding style and architecture decisions pretty well.
I also recommend in each session using Plan mode to get to a design you are happy with before generating any code.
The plan-build-test-reflect loop is equally important when using an LLM to generate code, as anyone who's seriously used the tech knows: if you yolo your way through a build without thought, it will collapse in on itself quickly. But if you DO apply that loop, you get to spend much more time on the part I personally enjoy, architecting the build and testing the resultant experience.
> While the LLMs get to blast through all the fun, easy work at lightning speed, we are then left with all the thankless tasks
This is, to me, the root of one disagreement I see playing out in every industry where AI has achieved any level of mastery. There's a divide between people who enjoy the physical experience of the work and people who enjoy the mental experience of the work. If the thinking bit is your favorite part, AI allows you to spend nearly all of your time there if you wish, from concept through troubleshooting. But if you like the doing, the typing, fiddling with knobs and configs, etc etc, all AI does is take the good part away.
Why would I want a fuzzy, vague, imprecise, up-to-interpretation programming language? I already have to struggle with that in documentation, specifications, peers and – of course – myself. Why would I take the one precise component and make it suffer from the same?
This contrasts of course with tasks such as search, where I'm not quite able to precisely express what I want. Here I find LLMs to be a fantastic advance. Same for e.g. operations between imprecise domains, like between natural languages.
Does this divide between "physical" and "mental" exist? Programming languages are formal languages that allow you to precisely and unambiguously express your ideas. I would say that "fiddling" with the code (as you say) is a kind of mental activity.
If there is actually someone out there that only dislikes AI coding assistants because they enjoy the physical act of typing and now have to do less of it (I have not seen this blog post yet), then I might understand your point.
Eh, physical and mental isn't the divide — it's more like people who enjoy code itself as a craft and people who simply see it as a means to an end (the application). Much like a writer might labor over their prose (the code) while telling a story (the application). Writing code is far more than the physical act of typing to those people.
Not forces, encourages.
A dev that spends an undue amount of time fiddling with knobs and configs probably sucks. Their mind isn't on the problem that needs to be solved.
Yeah I'm actually quite surprised that so many people are just telling AI to do X without actually developing a maintainable plan to do so first. It's no wonder that so many people are anti-vibe-coding — it's because their exposure to vibe coding is just telling Replit or Claude Code to do X.
I still do most of my development in my head, but I have a go-to prompt I ask Claude code when I'm stuck: "without writing any code, and maintaining existing patterns, tell me how to do X." it'd spit out some stuff, I'd converse with it to make sure it is a feasible solution that would work long term, then I tell it to execute the plan. But the process still starts in my head, not with a prompt.
I’ve found this concept trips CC up—- assertions are backwards, confusing comments in the test, etc. Just starting a prompt with “Use TDD to…” really helps.
I will raise that LLMs are pretty good at some of the non-coding tasks too.
eg. "I'm currently creating an AI for a turn based board game. Without doing any implementation, create a plan for the steps that need to be done including training off real world game data".
The LLM creates a tasklist for iterative steps to accomplish the above. It usually needs correction specific to the business/game needs but it's a great start and i recommend doing this just so the LLM has a doc with context on what its trying to achieve in a bigger picture as you have it complete tasks.
I think that this approach can already get us pretty far. One thing I'm missing is tooling to make it easier to build automation on top of, eg, Claude Code, but I'm sure it's going to come (and I'm tempted to try vibe coding it; if only I had the time).
LLMs aren't effective when used this way.
You still have to think.
IMO a vibe coder who is speaking their ideas to an agent which implements them is going to have way more time to think than a hand coder who is spending 80% of their time editing text.
obviously good and experienced engineers aren't going to be vibe coders/mollycoddlers by nature. but many good and experienced engineers will be pressured to make poor decisions by impatient business leaders. and that's the root of most AI anxiety: we all know it's going to be used as irresponsibly and recklessly as possible. it's not about the tech. it's about a system with broken incentives.
Which looks to attempt to give better "coding" understanding to the model instead of mere tokens and positioning and hence improve the coding capabilities of these "brilliant but unpredictable junior engineer" coding agents:
- https://ai.meta.com/research/publications/cwm-an-open-weight...
My one concrete pushback to the article is that it states the inevitable end result of vibe coding is a messy unmaintainable codebase. This is empirically not true. At this point I have many vibecoded projects that are quite complex but work perfectly. Most of these are for my private use but two of them serve in a live production context. It goes without saying that not only do these projects work, but they were accomplished 100x faster than I could have done by hand.
Do I also have vibecoded projects that went of the rails? Of course. I had to build those to learn where the edges of the model’s capabilities are, and what its failure modes are, so I can compensate. Vibecoding a good codebase is a skill. I know how to vibecode a good, maintainable codebase. Perhaps this violates your definition of vibecoding; my definition is that I almost never need to actually look at the code. I am just serving as a very hands-on manager. (Though I can look at the code if I need to - have 20 years of coding experience. But if I find that I need to look at the code, something has already gone badly wrong.)
Relevant anecdote: A couple of years ago I had a friend who was incredibly skilled at getting image models to do things that serious people asserted image models definitely couldn’t do at the time. At that time there were no image models that could get consistent text to appear in the image, but my friend could always get exactly the text you wanted. His prompts were themselves incredible works of art and engineering, directly grabbing hold of the fundamental control knobs of the model that most users are fumbling at.
Here’s the thing: any one of us can now make an image that is better than anything he was making at the time. Better compositionality, better understanding of intent, better text accuracy. We do this out of the box and without any attention paid to promoting voodoo at all. The models simply got that much better.
In a year or two, my carefully cultivated expertise around vibecoding will be irrelevant. You will get results like mine by just telling the model what you want. I assert this with high confidence. This is not disappointing to me, because I will be taking full advantage of the bleeding edge of capabilities throughout that period of time. Much like my friend, I don’t want to be good at managing AIs, I want to realize my vision.
Other than that, I keep hearing the same arguments - "LLMs free up more time for me to think about the 'important things'." Son, your system is not durable, your tests are misleading, and you can't reason about what's happening because you didn't write it. What important things are left to think about??
I cannot express how tired I am of seeing this beyond stupid take.
If you truly believe that, you have either only ever worked with the most piss-poor junior engineers, or you simply have never worked with junior engineers.
LLMs do not learn, LLMs do not ask clarifications, LLMs do not wonder if they're going in the wrong direction, LLMs do not have taste, LLMs do not have opinions, LLMs write absolutely illogical nonsense, LLMs do not ask themselves what is the best for the customer, LLMs have no context beyond what you have explicitely fed them for that one specific task, and much more.
I will say I always provide an initial context document about the feature/system, to avoid us starting with trivial questions. After about 45minutes I’ll often feel I’ve covered enough ground and given the problem enough thought to really put pen to paper. Off the back of this I’ll ask it to summarise the spec and produce a document. This can be a good point to ditch AI if you are so inclined but still get value from it.
Once the planning is done, the actual coding is very fast. The human review that follows is again slow, often also leading to minor new tickets.
The fact then when we read others' code, we don't remember/integrate it into our thinking as well as we do when we're the authors. So mentoring "AI Juniors" provides less growth then doing the job, esp. if it is mostly corrective actions.