> In retrospect, it made sense. Agents write units of changes that look good in isolation. They are consistent with themselves and your prompt. But respect for the whole, there is not. Respect for structural integrity there is not. Respect even for neighboring patterns there was not.
Well yea, but you can guard against this in several ways. My way is to understand my own codebase and look at the output of the LLM.
LLMs allow me to write code faster and it also gives a lot of discoverability of programming concepts I didn't know much about. For example, it plugged in a lot of Tailwind CSS, which I've never used before. With that said, it does not absolve me from not knowing my own codebase, unless I'm (temporarily) fine with my codebase being fractured conceptually in wonky ways.
I think vibecoding is amazing for creating quick high fidelity prototypes for a green field project. You create it, you vibe code it all the way until your app is just how you want it to feel. Then you refactor it and scale it.
I'm currently looking at 4009 lines of JS/JSX combined. I'm still vibecoding my prototype. I recently looked at the codebase and saw some ready made improvements so I did them. But I think I'll start to need to actually engineer anything once I reach the 10K line mark.
AI is incredibly dangerous because it can do the simple things very well, which prevents new programmers from learning the simple things ("Oh, I'll just have AI generate it") which then prevents them from learning the middlin' and harder and meta things at a visceral level.
I'm a CS teacher, so this is where I see a huge danger right now and I'm explicit with my students about it: you HAVE to write the code. You CAN'T let the machines write the code. Yes, they can write the code: you are a student, the code isn't hard yet. But you HAVE to write the code.
It doesn't PREVENT them from learning anything - said properly, it lets developers become lazy and miss important learning opportunities. That's not AIs fault.
I see junior devs hyping vibe coding and senior devs mostly using AI as an assistant. I fall in the latter camp myself.
I've hired and trained tons of junior devs out of university. They become 20x productive after a year of experience. I think vibe coding is getting new devs to 5x productivity, which seems amazing, but then they get stuck there because they're not learning. So after year one, they're a 5x developer, not a 20x developer like they should be.
I have some young friends who are 1-3 years into software careers I'm surprised by how little they know.
Not only that, it's constitution. I'm finding this with myself. After vibe coding for a month or so I let my subscription expire. Now when I look at the code it's like "ugh you mean now I have to think about this with my own brain???"
Even while vibe-coding, I often found myself getting annoyed just having to explain things. The amount of patience I have for anything that doesn't "just work" the first time has drifted toward zero. If I can't get AI to do the right thing after three tries, "welp, I guess this project isn't getting finished!"
It's not just laziness, it's like AI eats away at your pride of ownership. You start a project all hyped about making it great, but after a few cycles of AI doing the work, it's easy to get sucked into, "whatever, just make it work". Or better yet, "pretend to make it work, so I can go do something else."
Agreed. I think the divide is between code-as-thinking and code-as-implementation. Trivial assignments and toy projects and geeking out over implementation details are necessary to learn what code is, and what can be done with it. Otherwise your ideas are too vague to guide AI to an implementation.
Without the clarity that comes from thinking with code, a programmer using AI is the blind leading the blind.
The social aspect of a dialogue is relaxing, but very little improvement is happening. It's like a study group where one (relatively) incompetent student tries to advise another, and then test day comes and they're outperformed by the weirdo that worked alone.
Yes, exactly. I'm having a frustrating time reminding senior teachers of this, people with authority who should really know better. There seems to be some delusion that this technology will somehow change how people learn in a fundamental way.
The problem is: now they also need to learn to code with an LLM assistant. That goes beyond "coding it by yourself". Well, it's different, anyway. Another skill to teach.
Part of the issue here is that you can look at something and think "oh yeah I understand that, it makes perfect sense!", but then completely fail to reproduce it yourself.
This is a good point. Letting people learning to code to use AI, would be like letting 6 to 10 yo in school just use pocket calculators and not learn to do basic arithmetic manually. Yes IRL you will have a calculator at hand, yes, the calculator will make less mistakes, still, for you to learn und understand, you have to do it manually.
> Not only does an agent not have the ability to evolve a specification over a multi-week period as it builds out its lower components, it also makes decisions upfront that it later doesn’t deviate from.
That's your job.
The great thing about coding agents is that you can tell them "change of design: all API interactions need to go through a new single class that does authentication and retries and rate-limit throttling" and... they'll track down dozens or even hundreds of places that need updating and fix them all.
(And the automated test suite will help them confirm that the refactoring worked properly, because naturally you had them construct an automated test suite when they built those original features, right?)
Going back to typing all of the code yourself (my interpretation of "writing by hand") because you don't have the agent-managerial skills to tell the coding agents how to clean up the mess they made feels short-sighted to me.
No, that isn't. To quote your own blog, his job is to "deliver code [he's] proven to work", not to manage AI agents. The author has determined that managing AI agents is not an effective way to deliver code in the long term.
> you don't have the agent-managerial skills to tell the coding agents how to clean up the mess they made
The author has years of experience with AI assisted coding. Is there any way we can check to see if someone is actually skilled at using these tools besides whether they report/studies measure that they do better with them than without?
I think there is going to be an AI eternal summer. Both from developer to AI spec - where the AI implements to the spec to some level of quality, but then closing the gap after that is an endless chase of smaller items that don't all resolve at the same time. And from people getting frustrated with some AI implemented app, and so go off and AI implement another one, with a different set of features and failings.
I felt everything in this post quite emphatically until the “but I’m actually faster than the AI.”
Might be my skills but I can tell you right now I will not be as fast as the AI especially in new codebases or other languages or different environments even with all the debugging and hell that is AI pull request review.
I think the answer here is fast AI for things it can do on its own, and slow, composed, human in the loop AI for the bigger things to make sure it gets it right. (At least until it gets most things right through innovative orchestration and model improvement moving forward.)
I never trust the opinion of a single LLM model anymore - especially for more complex projects. I have seen Claude guarantee something is correct and then immediately apologize when I feed a critical review by Codex or Gemini. And, many times, the issues are not minor but are significant critical oversights by Claude.
My habit now: always get a 2nd or 3rd opinion before assuming one LLM is correct.
I came to "vibe coding" with an open mind, but I'm slowly edging in the same direction.
It is hands down good for code which is laborious or tedious to write, but once done, obviously correct or incorrect (with low effort inspection). Tests help but only if the code comes out nicely structured.
I made plenty of tools like this, a replacement REPL for MS-SQL, a caching tool in Python, a matplotlib helper. Things that I know 90% how to write anyway but don't have the time, but once in front of me, obviously correct or incorrect. NP code I suppose.
But business critical stuff is rarely like this, for me anyway. It is complex, has to deal with various subtle edge cases, be written defensively (so it fails predictably and gracefully), well structured etc. and try as I might, I can't get Claude to write stuff that's up to scratch in this department.
I'll give it instructions on how to write some specific function, it will write this code but not use it, and use something else instead. It will pepper the code with rookie mistakes like writing the same logic N times in different places instead of factoring it out. It will miss key parts of the spec and insist it did it, or tell me "Yea you are right! Let me rewrite it" and not actually fix the issue.
I also have a sense that it got a lot dumber over time. My expectations may have changed of course too, but still. I suspect even within a model, there is some variability of how much compute is used (eg how deep the beam search is) and supply/demand means this knob is continuously tuned down.
I still try to use Claude for tasks like this, but increasingly find my hit rate so low that the whole "don't write any code yet, let's build a spec" exercise is a waste of time.
I still find Claude good as a rubber duck or to discuss design or errors - a better Stack Exchange.
But you can't split your software spec into a set of SE questions then paste the code from top answers.
> The AI had simply told me a good story. Like vibewriting a novel, the agent showed me a good couple paragraphs that sure enough made sense and were structurally and syntactically correct. Hell, it even picked up on the idiosyncrasies of the various characters. But for whatever reason, when you read the whole chapter, it’s a mess. It makes no sense in the overall context of the book and the preceding and proceeding chapters.
This is the bit I think enthusiasts need to argue doesn't apply.
Have you ever read a 200 page vibewritten novel and found it satisfying?
So why do you think a 10 kLoC vibecoded codebase will be any good engineering-wise?
Everything the OP says can be true, but there’s a tipping point where you learn to break through the cruft and generate good code at scale.
It requires refactoring at scale, but GenAI is fast so hitting the same code 25 times isn’t a dealbreaker.
Eventually the refactoring is targeted at smaller and smaller bits until the entire project is in excellent shape.
I’m still working on Sharpee, an interactive fiction authoring platform, but it’s fairly well-baked at this point and 99% coded by Claude and 100% managed by me.
Sharpee is a complex system and a lot of the inner-workings (stdlib) were like coats of paint. It didn’t shine until it was refactored at least a dozen times.
It has over a thousand unit tests, which I’ve read through and refactored by hand in some cases.
The tale of the coder, who finds a legacy codebase (sometimes of their own making) and looks at it with bewilderment is not new. It's a curious one, to a degree, but I don't think it has much to do with vibe coding.
I have been working for 20 years and I haven’t really experienced this with any code I’ve written. Sure I don’t remember every line but I always recall the high level outlines.
The author also has multiple videos on his YouTube channel going over the specific issues hes had with AI that I found really interesting: https://youtube.com/@atmoio
Good luck finding an employer that lets you do this moving forward. The new reality is that no one can give the estimates they previously gave for tasks. \
"Amazingly, I’m faster, more accurate, more creative, more productive, and more efficient than AI, when you price everything in, and not just code tokens per hour."
I've gone through this cycle too, and what I realized is that as a developer a large part of your job is making sure the code you write works, is maintainable, and you can explain how it works.
I wish more critics would start to showcase examples of code slop. I'm not saying this because I defend the use of AI-coding, but rather because many junior devs. that read these types articles/blog posts may not know what slop is, or what it looks like. Simply put, you don't know what you don't know.
Karpathy coined the term vibecoding 11 months ago (https://x.com/karpathy/status/1886192184808149383). It caused quite a stir - because not only was it was a radically new concept, but fully agentic coding had only become recently possible. You've been vibe coding for two years??
My observation is that vibe-coded applications are significantly lower quality than traditional software. Anthropic software (which they claim to be 90% vibe coded) is extremely buggy, especially the UI.
130 comments
[ 2.9 ms ] story [ 90.9 ms ] threadhis points about why he stopped using AI: these are the things us reluctant AI adopters have been saying since this all started.
Well yea, but you can guard against this in several ways. My way is to understand my own codebase and look at the output of the LLM.
LLMs allow me to write code faster and it also gives a lot of discoverability of programming concepts I didn't know much about. For example, it plugged in a lot of Tailwind CSS, which I've never used before. With that said, it does not absolve me from not knowing my own codebase, unless I'm (temporarily) fine with my codebase being fractured conceptually in wonky ways.
I think vibecoding is amazing for creating quick high fidelity prototypes for a green field project. You create it, you vibe code it all the way until your app is just how you want it to feel. Then you refactor it and scale it.
I'm currently looking at 4009 lines of JS/JSX combined. I'm still vibecoding my prototype. I recently looked at the codebase and saw some ready made improvements so I did them. But I think I'll start to need to actually engineer anything once I reach the 10K line mark.
I'm a CS teacher, so this is where I see a huge danger right now and I'm explicit with my students about it: you HAVE to write the code. You CAN'T let the machines write the code. Yes, they can write the code: you are a student, the code isn't hard yet. But you HAVE to write the code.
I've hired and trained tons of junior devs out of university. They become 20x productive after a year of experience. I think vibe coding is getting new devs to 5x productivity, which seems amazing, but then they get stuck there because they're not learning. So after year one, they're a 5x developer, not a 20x developer like they should be.
I have some young friends who are 1-3 years into software careers I'm surprised by how little they know.
Even while vibe-coding, I often found myself getting annoyed just having to explain things. The amount of patience I have for anything that doesn't "just work" the first time has drifted toward zero. If I can't get AI to do the right thing after three tries, "welp, I guess this project isn't getting finished!"
It's not just laziness, it's like AI eats away at your pride of ownership. You start a project all hyped about making it great, but after a few cycles of AI doing the work, it's easy to get sucked into, "whatever, just make it work". Or better yet, "pretend to make it work, so I can go do something else."
Without the clarity that comes from thinking with code, a programmer using AI is the blind leading the blind.
The social aspect of a dialogue is relaxing, but very little improvement is happening. It's like a study group where one (relatively) incompetent student tries to advise another, and then test day comes and they're outperformed by the weirdo that worked alone.
People said this about compilers. It depends what layer you care to learn/focus on. AI at least gives us the option to move up another level.
That's your job.
The great thing about coding agents is that you can tell them "change of design: all API interactions need to go through a new single class that does authentication and retries and rate-limit throttling" and... they'll track down dozens or even hundreds of places that need updating and fix them all.
(And the automated test suite will help them confirm that the refactoring worked properly, because naturally you had them construct an automated test suite when they built those original features, right?)
Going back to typing all of the code yourself (my interpretation of "writing by hand") because you don't have the agent-managerial skills to tell the coding agents how to clean up the mess they made feels short-sighted to me.
No, that isn't. To quote your own blog, his job is to "deliver code [he's] proven to work", not to manage AI agents. The author has determined that managing AI agents is not an effective way to deliver code in the long term.
> you don't have the agent-managerial skills to tell the coding agents how to clean up the mess they made
The author has years of experience with AI assisted coding. Is there any way we can check to see if someone is actually skilled at using these tools besides whether they report/studies measure that they do better with them than without?
I have AI build self-contained, smallish tasks and I check everything it does to keep the result consistent with global patterns and vision.
I stay in the loop and commit often.
Looks to me like the problem a lot of people are having is that they have AI do the whole thing.
If you ask it "refactor code to be more modern", it might guess what you mean and do it in a way you like it or not, but most likely it won't.
If you keep tasks small and clearly specced out it works just fine. A lot better than doing it by hand in many cases, specially for prototyping.
Might be my skills but I can tell you right now I will not be as fast as the AI especially in new codebases or other languages or different environments even with all the debugging and hell that is AI pull request review.
I think the answer here is fast AI for things it can do on its own, and slow, composed, human in the loop AI for the bigger things to make sure it gets it right. (At least until it gets most things right through innovative orchestration and model improvement moving forward.)
this is such an individualized technology that two people at the same starting point two years ago, could've developed wildly different workflows.
My habit now: always get a 2nd or 3rd opinion before assuming one LLM is correct.
It is hands down good for code which is laborious or tedious to write, but once done, obviously correct or incorrect (with low effort inspection). Tests help but only if the code comes out nicely structured.
I made plenty of tools like this, a replacement REPL for MS-SQL, a caching tool in Python, a matplotlib helper. Things that I know 90% how to write anyway but don't have the time, but once in front of me, obviously correct or incorrect. NP code I suppose.
But business critical stuff is rarely like this, for me anyway. It is complex, has to deal with various subtle edge cases, be written defensively (so it fails predictably and gracefully), well structured etc. and try as I might, I can't get Claude to write stuff that's up to scratch in this department.
I'll give it instructions on how to write some specific function, it will write this code but not use it, and use something else instead. It will pepper the code with rookie mistakes like writing the same logic N times in different places instead of factoring it out. It will miss key parts of the spec and insist it did it, or tell me "Yea you are right! Let me rewrite it" and not actually fix the issue.
I also have a sense that it got a lot dumber over time. My expectations may have changed of course too, but still. I suspect even within a model, there is some variability of how much compute is used (eg how deep the beam search is) and supply/demand means this knob is continuously tuned down.
I still try to use Claude for tasks like this, but increasingly find my hit rate so low that the whole "don't write any code yet, let's build a spec" exercise is a waste of time.
I still find Claude good as a rubber duck or to discuss design or errors - a better Stack Exchange.
But you can't split your software spec into a set of SE questions then paste the code from top answers.
This is the bit I think enthusiasts need to argue doesn't apply.
Have you ever read a 200 page vibewritten novel and found it satisfying?
So why do you think a 10 kLoC vibecoded codebase will be any good engineering-wise?
It requires refactoring at scale, but GenAI is fast so hitting the same code 25 times isn’t a dealbreaker.
Eventually the refactoring is targeted at smaller and smaller bits until the entire project is in excellent shape.
I’m still working on Sharpee, an interactive fiction authoring platform, but it’s fairly well-baked at this point and 99% coded by Claude and 100% managed by me.
Sharpee is a complex system and a lot of the inner-workings (stdlib) were like coats of paint. It didn’t shine until it was refactored at least a dozen times.
It has over a thousand unit tests, which I’ve read through and refactored by hand in some cases.
The results speak for themselves.
https://sharpee.net/ https://github.com/chicagodave/sharpee/
It’s still in beta, but not far from release status.
I admit I could be an outlier though.
"Amazingly, I’m faster, more accurate, more creative, more productive, and more efficient than AI, when you price everything in, and not just code tokens per hour."
For 99.99% of developers this just won't be true.
"AI can be good -- very good -- at building parts. For now, it's very bad at the big picture."