I vibe coded a retro emulator and assembler with tests. Prompts were minimal and I got really great results (Gemini 3). I tried vibe coding the tricky proprietary part of an app I worked on a few years ago; highly technical domain (yes vague don’t care to dox myself). Lots of prompting and didn’t get close.
There are literally thousands of retro emulators on github. What I was trying to do had zero examples on GitHub. My take away is obvious as of now. Some stuff is easy some not at all.
I dunno I get it to do stuff every day that’s never been done before, if you prompt really well, give loads of context, and take it slowly it’s amazing at it and still saves me a ton of time.
I always suspect the devil is in the details with these posts. The difference between smart prompting strategies and the way I see most people prompt ai is vast.
If the "hard part" is writing a detailed spec for the code you're about to commit to the project, AI can actually help you with that if you tell it to. You just can't skip that part of the work altogether and cede all control to a runaway slop generator.
It's pretty difficult to say what it's going to be three months from now. A few months ago Gemini 2.x in IDEA and related IDEs had to be dragged through coding tasks and would create dumb build time errors on its way to making buggy code.
Gemini in Antigravity today is pretty interesting, to the point where it's worth experimenting with vague prompts just to see what it comes up with.
Coding agents are not going to just change coding. They make a lot of detailed product management work obsolete and smaller team sizes will make it imperative to reread the agile manifesto and and discard scrum dogma.
as usual the last 20% need 80% and the other 80% need 20% but my god did Ai make my bs corpo easy repeatable shit work like skimming docs writing summaries, skimming jira confluence and so on actually easier and for 90% of bs crud app changes the first draft is also already pretty good tbh I don't write hard/difficult code more then once a week/month.
Also re: "I spent longer arguing with the agent and recovering the file than I would have spent writing the test myself."
In my humble experience arguing with an LLM is a waste of time, and no-one should be spending time recovering files. Just do small changes one at a time, commit when you get something working, and discard your changes and try again if it doesn't.
I don't think AI is a panacea, it's just knowing when it's the right tool for the job and when it isn't.
Daily agentic user here, and to me the problem here is the very notion of "vibe coding". If you're even thinking in those terms - this idea that never looking at the code has become a goal unto itself - then IMO you're doing LLM-assisted development wrong.
This is very much a hot take, but I believe that Claude Code and its yolo peers are an expensive party trick that gives people who aren't deep into this stuff an artificially negative impression of tools that can absolutely be used in a responsible, hugely productive way.
Seriously, every time I hear anecdotes about CC doing the sorts of things the author describes, I wonder why the hell anyone is expecting more than quick prototypes from an LLM running in a loop with no intervention from an experienced human developer.
Vibe coding is riding your bike really fast with your hands off the handles. It's sort of fun and feels a bit rebellious. But nobody who is really good at cycling is talking about how they've fully transitioned to riding without touching the handles, because that would be completely stupid.
We should feel the same way about vibe coding.
Meanwhile, if you load up Cursor and break your application development into bite sized chunks, and then work through those chunks in a sane order using as many Plan -> Agent -> Debug conversations with Opus 4.5 (Thinking) as needed, you too will obtain the mythical productivity multipliers you keep accusing us of hallucinating.
1. Allowing non-developers to provide very detailed specs for the tools they want or experiences they are imagining
2. Allowing developers to write code using frameworks/languages they only know a bit of and don't like; e.g. I use it to write D3 visualizations or PNG extracts from datastores all the time, without having to learn PNG API or modern javascript frameworks. I just have to know enough to look at the console.log / backtrace and figure out where the fix can be.
3. Analysing large code bases for specific questions (not as accurate on "give me an overall summary" type questions - that one weird thing next to 19 normal things doesn't stick in its craw as much as for a cranky human programmer.
It does seem to benefit cranking thru a list of smallish features/fixes rapidly, but even 4.5 or 4.6 seem to get stuck in weird dead ends rarely enough that I'm not expecting it, but often enough to be super annoying.
I've been playing around with Gas Town swarming a large scale Java migration project, and its been N declarations of victory and still mvn test isn't even compiling. (mvn build is ok, and the pom is updated to the new stack, so it's not nothing). (These are like 50/50 app code/test code repos).
Why do all of that when you can just keep a tight hold on an agent that is operating at the speed that you can think about what you're actually doing?
Again, if you're just looking to spend a lot of money on the party trick, don't let me yuck your yum. It just seems like doing things in a way that is almost guaranteed to lead to the outcomes that people love to complain aren't very good.
As someone getting excellent results on a huge (550k LoC) codebase only because I'm directing every feature, my bottleneck is always going to be the speed at which I can coherently describe what needs to be done + a reasonable amount of review to make sure that what happened is what I was looking for. This can only work because I explicitly go through a planning cycle before handing it to the agent.
I feel like if you consider understanding what your LLM is doing for you to be unacceptably slow and burdensome, then you deserve exactly what you're going to get out of this process.
Totally agree on ai assisted coding resulting in randomly changed code. Sometimes it’s subtle and other times entire methods are removed. I have moved back to just using a JetBrains IDE and coping files in to Gemini so that I can limit context. Then I use the IDE to inspect changes in a git diff, regression test everything, and after all that, commit.
I think AI is just a massive force multiplier. If your codebase has bad foundation and going in the wrong direction with lots of hacks, it will just write code which mirrors the existing style... And you get exactly was OP is suggesting.
If however, your code foundations are good and highly consistent and never allow hacks, then the AI will maintain that clean style and it becomes shockingly good; in this case, the prompting barely even matters. The code foundation is everything.
But I understand why a lot of people are still having a poor experience. Most codebases are bad. They work (within very rigid constraints, in very specific environments) but they're unmaintainable and very difficult to extend; require hacks on top of hacks. Each new feature essentially requires a minor or major refactoring; requiring more and more scattered code changes as everything is interdependent (tight coupling, low cohesion). Productivity just grinds to a slow crawl and you need 100 engineers to do what previously could have been done with just 1. This is not a new effect. It's just much more obvious now with AI.
I've been saying this for years but I think too few engineers had actually built complex projects on their own to understand this effect. There's a parallel with building architecture; you are constrained by the foundation of the building. If you designed the foundation for a regular single storey house, you can't change your mind half-way through the construction process to build a 20-storey skyscraper. That said, if your foundation is good enough to support a 100 storey skyscraper, then you can build almost anything you want on top.
My perspective is if you want to empower people to vibe code, you need to give them really strong foundations to work on top of. There will still be limitations but they'll be able to go much further.
My experience is; the more planning and intelligence goes into the foundation, the less intelligence and planning is required for the actual construction.
I just did my first “AI native coding project”. Both because for now I haven’t run into any quotas using Codex CLI with my $20/month ChatGPT subscription and the company just gave everyone an $800/month Claude allowance.
Before I even started the implementation I:
1. Put the initial sales contract with the business requirements.
2. Notes I got from talking to sales
3. The transcript of the initial discovery calls
4. My design diagrams that were well labeled (cloud architecture and what each lambda does)
5. The transcript of the design review and my explanations and answering questions.
6. My ChatGPT assisted breakdown of the Epics/stories and tasks I had to do for the PMO
I then told ChatGPT to give a detailed breakdown of everything during the session as Markdown
That was the start of my AGENTS.md file.
While working through everything task by task and having Codex/Claude code do the coding, I told it to update a separate md file with what it did and when I told it to do something differently and why.
Any developer coming in after me will have complete context of the project from the first git init and they and the agents will know the why behind every decision that was made.
Can you say that about any project that was done before GenAI?
This does not track with my experience, trying agents out in a ~100K LOC codebase written exclusively by me. I can't tell you whether nor not it has a good foundation by your standards, but I find the outputs to be tasteless, and there should be more than enough context for what the style of the code is.
Given how adamant some people I respect a lot are about how good these models are, I was frankly shocked to see SOA models do transformations like
BEFORE:
// 20 lines
AFTER
if (something)
// the 20 lines
else
// the same 20 lines, one boolean changed in the middle
When I point this out, it extracts said 20 lines into a function that takes in the entire context used in the block as arguments:
AFTER 2:
if (something)
function_that_will_never_be_used_anywhere_else(a, b, c, &d, &e, &f, true);
else
function_that_will_never_be_used_anywhere_else(a, b, c, &d, &e, &f, false);
It also tends to add these comments that don't document anything, but rather just describe the latest change it did to the code:
// Extracted repeating code into a function:
void function_that_will_never_be_used_anywhere_else(...) {
...
}
and to top it off it has the audacity to tell me "The code is much cleaner now. Happy building! (rocketship emoji)"
Training is the process of regressing to the mean with respect to the given data. It's no surprise that it wears away sharp corners and inappropriately fills recesses of collective knowledge in the act of its reproduction.
The pattern matching and absence or real thinking is still strong.
Tried to move some excel generation logic from epplus to closedxml library.
ClosedXml has basically the same API so the conversion was successful. Not a one-shot but relatively easy with a few manual edits.
But closedxml has no batch operations (like apply style to the entire column): the api is there but internal implementation is on cell after cell basis. So if you have 10k rows and 50 columns every style update is a slow operaton.
Naturally, told all about this to codex 5.3 max thinking level. The fucker still succumbed to range updates here and there.
Told it explicitly to make a style cache and reuse styles on cells on same y axis.
5-6 attempts — fucker still tried ranges here and there. Because that is what is usually done.
It seems like a big part of the divide is that people who learned software engineering find vibe coding to be unsuitable for any project intended to be in use for more than a few while those who learned coding think vibe coding is the next big thing because they never have to deal with the consequences of the bad code.
Yep it is why the work getting over the threshold is just as long as it was without AI.
Someone mentioned it is a force multiplier I don't disagree with this, it is a force multiplier in the mundane and ordinary execution of tasks. Complex ones get harder and hard for it where humans visualize the final result where AI can't. It is predicting from input but it can't know the destination output if the destination isn't part of the input.
I don't think it makes any part harder. What it does do is expose what people have ignored their whole career: the hard part. The last 15 years of software development has been 'human vibe coding'; copy+pasting snippets from SO without understanding them, no planning, constant rearchitecting, shipping code to prod as long as it runs on your laptop. Now that the AI is doing it, suddenly people want to plan their work and enforce tests? Seems like a win-win to me. Even if it slows down development, that would be a win, because the result is enforcement of better quality.
Well said. Much like the self driving debate we don’t need them to be perfect, just better than us to be useful, and clearly they already are for the most part.
I'm working on a paper connecting articulatory phonology to soliton physics. Speech gestures survive coarticulatory overlap the same way solitons survive collision. The nonlinear dynamics already in the phonetics literature are structurally identical to soliton equations. Nobody noticed because these fields don't share conferences.
The article's easy/hard distinction is right but the ceiling for "hard" is too low. The actually hard thing AI enables isn't better timezone bug investigation LOL! It's working across disciplinary boundaries no single human can straddle.
Some time back, my manager at the time, who shall remain nameless told the group that having AI is like having 10 people work for you ( he actually had a slightly smaller number, but it was said almost word for word like in the article ) with the expectation being set as: 'you should now be able to do 10x as much'.
Needless to say, he was wrong and gently corrected over the course of time. In his defense, his use cases for LLMs at the time were summarizing emails in his email client.. so..eh.. not exactly much to draw realistic experience from.
I hate to say it, but maybe nvidia CEO is actually right for once. We have a 'new smart' coming to our world. The type of a person that can move between worlds of coding, management, projects and CEOing with relative ease and translate between those worlds.
People need to consider / realize that the vast majority of source code training data is Github, Gitlab, and essentially the huge sea of started, maybe completed, student and open source project. That large body of source code is for the most part unused, untested, and unsuccessful software of unknown quality. That source code is AI's majority training data, and an AI model in training has no idea what is quality software and what is "bad" software. That means the average source code generated by AI not necessarily good software. Considering it is an average of algorithms, it's surprising generated code runs at all. But then again, generating compiling code is actually trainable, so what is generated can receive extra training support. However, that does not improve the quality of the source code training data, just the fact that it will compile.
> huge sea of started, maybe completed, student and open source project.
Which is easy to filter out based on downloads, version numbering, issue tracker entries, and wikipedia or other external references if the project is older and archived, but historically noteworthy (like the source code for Netscape Communicator or DOOM).
> On a personal project, I asked an AI agent to add a test to a specific file. The file was 500 lines before the request and 100 lines after. I asked why it deleted all the other content. It said it didn't. Then it said the file didn't exist before. I showed it the git history and it apologised, said it should have checked whether the file existed first.
Ha! Yesterday an agent deleted the plan file after I told it to "forget about it" (as in, leave it alone).
The "marathon of sprints" paradigm is now everywhere and AI is turning it to 120%. I am not sure how many devs can keep sprinting all the time without any rest. AI maybe can help but it tends to go off-rails quickly when not supervised and reading code one did not author is more exhausting than just fixing one's own code.
I think the author answers their own question at the end.
The first 3/4 of the article is "we must be responsible for every line of code in the application, so having the LLM write it is not helping".
The last 1/4 is "we had an urgent problem so we got the LLM to look at the code base and find the solution".
The situation we're moving to is that the LLM owns the code. We don't look at the code. We tell the LLM what is needed, and it writes the code. If there's a bug, we tell the LLM what the bug is, and the LLM fixes it. We're not responsible for every line of code in the application.
It's exactly the same as with a compiler. We don't look at the machine code that the compiler produces. We tell the compiler what we want, using a higher-level abstraction, and the compiler turns that into machine code. We trust compilers to do this error-free, because 50+ years of practice has proven to us that they do this error-free.
We're maybe ~1 year into coding agents. It's not surprising that we don't trust LLMs yet. But we will.
And it's going to be fascinating how this changes the Computer Science. We have interpreted languages because compilers got so good. Presumably we'll get to non-human-readable languages that only LLMs can use. And methods of defining systems to an LLM that are better than plain English.
71 comments
[ 2.2 ms ] story [ 78.0 ms ] threadThere are literally thousands of retro emulators on github. What I was trying to do had zero examples on GitHub. My take away is obvious as of now. Some stuff is easy some not at all.
I always suspect the devil is in the details with these posts. The difference between smart prompting strategies and the way I see most people prompt ai is vast.
Gemini in Antigravity today is pretty interesting, to the point where it's worth experimenting with vague prompts just to see what it comes up with.
Coding agents are not going to just change coding. They make a lot of detailed product management work obsolete and smaller team sizes will make it imperative to reread the agile manifesto and and discard scrum dogma.
Also re: "I spent longer arguing with the agent and recovering the file than I would have spent writing the test myself."
In my humble experience arguing with an LLM is a waste of time, and no-one should be spending time recovering files. Just do small changes one at a time, commit when you get something working, and discard your changes and try again if it doesn't.
I don't think AI is a panacea, it's just knowing when it's the right tool for the job and when it isn't.
This is very much a hot take, but I believe that Claude Code and its yolo peers are an expensive party trick that gives people who aren't deep into this stuff an artificially negative impression of tools that can absolutely be used in a responsible, hugely productive way.
Seriously, every time I hear anecdotes about CC doing the sorts of things the author describes, I wonder why the hell anyone is expecting more than quick prototypes from an LLM running in a loop with no intervention from an experienced human developer.
Vibe coding is riding your bike really fast with your hands off the handles. It's sort of fun and feels a bit rebellious. But nobody who is really good at cycling is talking about how they've fully transitioned to riding without touching the handles, because that would be completely stupid.
We should feel the same way about vibe coding.
Meanwhile, if you load up Cursor and break your application development into bite sized chunks, and then work through those chunks in a sane order using as many Plan -> Agent -> Debug conversations with Opus 4.5 (Thinking) as needed, you too will obtain the mythical productivity multipliers you keep accusing us of hallucinating.
1. Allowing non-developers to provide very detailed specs for the tools they want or experiences they are imagining
2. Allowing developers to write code using frameworks/languages they only know a bit of and don't like; e.g. I use it to write D3 visualizations or PNG extracts from datastores all the time, without having to learn PNG API or modern javascript frameworks. I just have to know enough to look at the console.log / backtrace and figure out where the fix can be.
3. Analysing large code bases for specific questions (not as accurate on "give me an overall summary" type questions - that one weird thing next to 19 normal things doesn't stick in its craw as much as for a cranky human programmer.
It does seem to benefit cranking thru a list of smallish features/fixes rapidly, but even 4.5 or 4.6 seem to get stuck in weird dead ends rarely enough that I'm not expecting it, but often enough to be super annoying.
I've been playing around with Gas Town swarming a large scale Java migration project, and its been N declarations of victory and still mvn test isn't even compiling. (mvn build is ok, and the pom is updated to the new stack, so it's not nothing). (These are like 50/50 app code/test code repos).
Why do all of that when you can just keep a tight hold on an agent that is operating at the speed that you can think about what you're actually doing?
Again, if you're just looking to spend a lot of money on the party trick, don't let me yuck your yum. It just seems like doing things in a way that is almost guaranteed to lead to the outcomes that people love to complain aren't very good.
As someone getting excellent results on a huge (550k LoC) codebase only because I'm directing every feature, my bottleneck is always going to be the speed at which I can coherently describe what needs to be done + a reasonable amount of review to make sure that what happened is what I was looking for. This can only work because I explicitly go through a planning cycle before handing it to the agent.
I feel like if you consider understanding what your LLM is doing for you to be unacceptably slow and burdensome, then you deserve exactly what you're going to get out of this process.
If however, your code foundations are good and highly consistent and never allow hacks, then the AI will maintain that clean style and it becomes shockingly good; in this case, the prompting barely even matters. The code foundation is everything.
But I understand why a lot of people are still having a poor experience. Most codebases are bad. They work (within very rigid constraints, in very specific environments) but they're unmaintainable and very difficult to extend; require hacks on top of hacks. Each new feature essentially requires a minor or major refactoring; requiring more and more scattered code changes as everything is interdependent (tight coupling, low cohesion). Productivity just grinds to a slow crawl and you need 100 engineers to do what previously could have been done with just 1. This is not a new effect. It's just much more obvious now with AI.
I've been saying this for years but I think too few engineers had actually built complex projects on their own to understand this effect. There's a parallel with building architecture; you are constrained by the foundation of the building. If you designed the foundation for a regular single storey house, you can't change your mind half-way through the construction process to build a 20-storey skyscraper. That said, if your foundation is good enough to support a 100 storey skyscraper, then you can build almost anything you want on top.
My perspective is if you want to empower people to vibe code, you need to give them really strong foundations to work on top of. There will still be limitations but they'll be able to go much further.
My experience is; the more planning and intelligence goes into the foundation, the less intelligence and planning is required for the actual construction.
I just did my first “AI native coding project”. Both because for now I haven’t run into any quotas using Codex CLI with my $20/month ChatGPT subscription and the company just gave everyone an $800/month Claude allowance.
Before I even started the implementation I:
1. Put the initial sales contract with the business requirements.
2. Notes I got from talking to sales
3. The transcript of the initial discovery calls
4. My design diagrams that were well labeled (cloud architecture and what each lambda does)
5. The transcript of the design review and my explanations and answering questions.
6. My ChatGPT assisted breakdown of the Epics/stories and tasks I had to do for the PMO
I then told ChatGPT to give a detailed breakdown of everything during the session as Markdown
That was the start of my AGENTS.md file.
While working through everything task by task and having Codex/Claude code do the coding, I told it to update a separate md file with what it did and when I told it to do something differently and why.
Any developer coming in after me will have complete context of the project from the first git init and they and the agents will know the why behind every decision that was made.
Can you say that about any project that was done before GenAI?
Given how adamant some people I respect a lot are about how good these models are, I was frankly shocked to see SOA models do transformations like
When I point this out, it extracts said 20 lines into a function that takes in the entire context used in the block as arguments: It also tends to add these comments that don't document anything, but rather just describe the latest change it did to the code: and to top it off it has the audacity to tell me "The code is much cleaner now. Happy building! (rocketship emoji)"Tried to move some excel generation logic from epplus to closedxml library.
ClosedXml has basically the same API so the conversion was successful. Not a one-shot but relatively easy with a few manual edits.
But closedxml has no batch operations (like apply style to the entire column): the api is there but internal implementation is on cell after cell basis. So if you have 10k rows and 50 columns every style update is a slow operaton.
Naturally, told all about this to codex 5.3 max thinking level. The fucker still succumbed to range updates here and there.
Told it explicitly to make a style cache and reuse styles on cells on same y axis.
5-6 attempts — fucker still tried ranges here and there. Because that is what is usually done.
Not here yet. Maybe in a year. Maybe never.
Someone mentioned it is a force multiplier I don't disagree with this, it is a force multiplier in the mundane and ordinary execution of tasks. Complex ones get harder and hard for it where humans visualize the final result where AI can't. It is predicting from input but it can't know the destination output if the destination isn't part of the input.
The article's easy/hard distinction is right but the ceiling for "hard" is too low. The actually hard thing AI enables isn't better timezone bug investigation LOL! It's working across disciplinary boundaries no single human can straddle.
Needless to say, he was wrong and gently corrected over the course of time. In his defense, his use cases for LLMs at the time were summarizing emails in his email client.. so..eh.. not exactly much to draw realistic experience from.
I hate to say it, but maybe nvidia CEO is actually right for once. We have a 'new smart' coming to our world. The type of a person that can move between worlds of coding, management, projects and CEOing with relative ease and translate between those worlds.
Which is easy to filter out based on downloads, version numbering, issue tracker entries, and wikipedia or other external references if the project is older and archived, but historically noteworthy (like the source code for Netscape Communicator or DOOM).
Ha! Yesterday an agent deleted the plan file after I told it to "forget about it" (as in, leave it alone).
I mean in a 'tistic kind of way that makes perfect sense.
The first 3/4 of the article is "we must be responsible for every line of code in the application, so having the LLM write it is not helping".
The last 1/4 is "we had an urgent problem so we got the LLM to look at the code base and find the solution".
The situation we're moving to is that the LLM owns the code. We don't look at the code. We tell the LLM what is needed, and it writes the code. If there's a bug, we tell the LLM what the bug is, and the LLM fixes it. We're not responsible for every line of code in the application.
It's exactly the same as with a compiler. We don't look at the machine code that the compiler produces. We tell the compiler what we want, using a higher-level abstraction, and the compiler turns that into machine code. We trust compilers to do this error-free, because 50+ years of practice has proven to us that they do this error-free.
We're maybe ~1 year into coding agents. It's not surprising that we don't trust LLMs yet. But we will.
And it's going to be fascinating how this changes the Computer Science. We have interpreted languages because compilers got so good. Presumably we'll get to non-human-readable languages that only LLMs can use. And methods of defining systems to an LLM that are better than plain English.