> ... I’ll keep pulling PRs locally, adding more git hooks to enforce code quality, and zooming through coding tasks—only to realize ChatGPT and Claude hallucinated library features and I now have to rip out Clerk and implement GitHub OAuth from scratch.
I don't get this, how many git hooks do you need to identify that Claude had hallucinated a library feature? Wouldn't a single hook running your tests identify that?
Well yeah, as the app scales it will bump up against context limits. Giving it sandboxed areas to do specific tasks will speed it up again, but that’s not possible with everything.
Maybe I’ve misunderstood this, so correct me if I’m wrong… do actual professional developers let enough code be generated to include entire libraries that handle things as important as authentication, and then build on top of it without making sure the previously generated code actually does what it’s supposed to? Just accept local PRs written by AI, with a very sternly worded “now you better not make any bullshit” system prompt? All this just in time to ramp up AI penetration tools. Jesus.
It’s kind of crazy to me how the cool kid take on software development, as recent as 3 years ago, was: strictly-typed everything, ‘real men’ don’t use garbage collection, everything must be optimized to death even when it isn’t really necessary, etc. and now it seems to be ‘you don’t seriously expect me to look at ‘every single line of code’ I submit, do you?’
AI tools seem excellent at getting through boilerplate stuff at the start of a project. But as time goes on and you have to think about what you are doing, it'll be faster to write it yourself than to convey it in natural language to an LLM. I don't see this as an issue with the tool, but just getting a better idea of what it is really good for.
I tend to think of it as magical incantations rather than boilerplate, and the AI is Harry Potter, except in this case Harry is a pathological liar & psychopath - but a useful one...
Even it's slow, you can run multiple agents. You can have one doing changes, while another writes documentation, while another does security checks, while another looks for optimizations. Persist finding to markdown files to track progress and for cross-agent knowledge sharing if need. And do whatever else while it's all running. This has been my experience.
My employer hosts one of the largest Ruby on Rails apps in the world. I've noticed that Claude Code takes a long time to grep for what it needs. Cursor is much better at this (probably because of local project indexing). Due to this, I favor Cursor over CC in my day to day workflows. In smaller code bases, both are pretty fast.
When building a project from scratch using AI, it can be tempting to give in to the vibe and ignore the structure/architecture and let it evolve naturally. This is a bad idea when humans do it, and it's also a bad idea when LLM agents do it. You have to be considering architecture, dataflow, etc from the beginning, and always stay on top of it without letting it drift.
I have tried READMEs scattered through the codebase but I still have trouble keeping the agent aware of the overall architecture we built.
I've never done QA.
Just thinking about doing QA makes my head swirl.
But yes, because of LLMs I am now a part time QA engineer, and I think that it's kinda helping me be a better developer.
Im working on a massive feature at work, something I can't just give to an agent and I already feel like something changed in how I think about every little piece of code im adding. didn't see that coming.
Somewhat related, I Found cursor/VS was slowing to the point of being unusable. Turning on privacy mode helped, but the main culprit was extremely verbose logging. Running `fatrace -c --command=cursor` discovered the issue.
The disk in question was an HDD and the problem disappeared (or is better hidden) after symlinking the log dir to an SSD.
As for code itself, I've never had an issue with slowness. If anything it's the verbosity of wanting to explain itself and excess logging in the code it creates.
I've found LLMs to be very good at writing design docs and finding problems in code.
Currently they're better at locating problems than fixing them without direction. Gemini seems smarter and better at architecture and best practices. Claude seems dumber but is more focused on getting things done.
The right solution is going to be a variety of tools and LLMs interacting with each other. But it's going to take real humans having real experience with LLMs to get there. It's not something that you can just dream up on paper and have it work out well since it depends so much on the details of the current models.
This should be called the eternal, unbearable slowness of code review, because the author writes that the AI actually churns out code extremely rapidly. The (hopefully capable, attentive, careful) human is the bottleneck here, as it should be
I'm still calibrating myself on the size of task that I can get Claude Code to do before I have to intervene.
I call this problem the "goldilocks" problem. The task has to be large enough that it outweighs the time necessary to write out a sufficiently detailed specification AND to review and fix the output. It has to be small enough that Claude doesn't get overwhelmed.
The issue with this is, writing a "sufficiently detailed specification" is task dependent. Sometimes a single sentence is enough, other times a paragraph or two, sometimes a couple of pages is necessary. And the "review and fix" phase again is totally dependent and completely unknown. I can usually estimate the spec time but the review and fix phase is a dice roll dependent on the output of the agent.
And the "overwhelming" metric is again not clear. Sometimes Claude Code can crush significant tasks in one shot. Other times it can get stuck or lost. I haven't fully developed an intuition for this yet, how to differentiate these.
What I can say, this is an entirely new skill. It isn't like architecting large systems for human development. It isn't like programming. It is its own thing.
> What I can say, this is an entirely new skill. It isn't like architecting large systems for human development. It isn't like programming. It is its own thing.
It's management!
I find myself asking very similar questions to you: how much detail is too much? How likely is this to succeed without my assistance? If it does succeed, will I need to refactor? Am I wasting my time delegating or should I just do it?
It's almost identical to when I delegate a task to a junior... only the feedback cycle of "did I guess correctly here" is a lot faster... and unlike a junior, the AI will never get better from the experience.
type safety, integration testing, and thorough readmes are now cheap, I don't know why any developer would not be using them with claude code. even if all the LLM services go under tomorrow you'll still have code that practically autocompletes itself.
Gemini CLI is pretty weak, but the Gemini 2.5 pro is still the best for long contexts. Claude is great but it crumbles as you start to get in the 50-100k range. I find Gemini doesn't start to crack until the 150-200k range. It's too bad the tooling around it is mediocre at best.
This illustrates a fundamental truth of maintaining software with LLMs: While programmers can use LLMs to produce huge amounts of code in a short time, they still need to read and understand it. It is simply not possible to delegate understanding a huge codebase to an AI, at least not yet.
In my experience, the real "pain" of programming lies in forcing yourself to absorb a flood of information and connecting the dots. Writing code is, in many ways, like taking a walk: you engage in a cognitively light activity that lets ideas shuffle, settle, and mature in the background.
When LLMs write all the code for you, you lose that essential mental rest. The quiet moments where you internalize concepts, spot hidden bugs, and develop a mental map of the system.
Prompting it better during development can really help here.
I have an emerging workflow orchestrated by Claude Code custom commands and subagents that turns even an informal description of a feature into a full fledged PRD, then an "architect" command researches and produces a well thought out and documented technical design. I can review that design document and then give it to the "planner" command, which breaks it down into Phases and Tasks. Then I have a "developer" command iterate through through and implement the Phases one by one. After each phase it runs a detailed code review using my "review" subagent.
Since I've started using this document-driven, guided workflow I've seen quality of the output noticeably improve.
I've no idea why, but the phrase "it's addicting" is really annoying, I'm pretty certain it should "it's addictive". I've started seeing it everywhere. (Note, I haven't completely lost my mind, it's in that article).
You can prompt Claude into (assess -> fix -> validate) -> assess -> ... loops pretty easily. You can do this with unit test coverage, and sometimes it's nice to come back to 100% coverage of the codebase (and the code review agent kept the tests from being hot garbage). You can push this with front end tests, playwright, etc to really get deep into validating your application without actually slogging through PRs (yet!).
My pattern with claude code is to let stuff simmer in the background with a detailed PRD, and just turn the screws with progressively more testing and type checking. I'll use repomix to put my entire codebase into gemini 2.5 pro, chat with it for a bit and then ask it to generate a highly detailed work plan for claude code to make the codebase more production hardened/launch ready. If I don't burn my plan tokens first, that gemini prompt can keep claude running for like ~3 hours usually. If you repeat this gemini plan -> claude implement step a few times gemini will eventually start to tell you to stop being a chicken and launch your great app.
Mistral may not be the smartest chat assistant out there, but I've stopped using others entirely given how slow they are compared to Mistral (which runs inference with Cerebras).
Waiting for an AI to complete its task isn't a fun thing at all, and I'd chose the fast 70% correct response any day over the slow 90% correct one. Because by the time the slow one gives you its first attempt, you'd have clarified you need and fixed the output from the fast one.
Sure if we get to the point where the slow system is 100% right, then it's no big deal if it's slow, but we're still far from that point.
36 comments
[ 4.4 ms ] story [ 30.2 ms ] threadI don't get this, how many git hooks do you need to identify that Claude had hallucinated a library feature? Wouldn't a single hook running your tests identify that?
It’s kind of crazy to me how the cool kid take on software development, as recent as 3 years ago, was: strictly-typed everything, ‘real men’ don’t use garbage collection, everything must be optimized to death even when it isn’t really necessary, etc. and now it seems to be ‘you don’t seriously expect me to look at ‘every single line of code’ I submit, do you?’
I have tried READMEs scattered through the codebase but I still have trouble keeping the agent aware of the overall architecture we built.
The disk in question was an HDD and the problem disappeared (or is better hidden) after symlinking the log dir to an SSD.
As for code itself, I've never had an issue with slowness. If anything it's the verbosity of wanting to explain itself and excess logging in the code it creates.
Currently they're better at locating problems than fixing them without direction. Gemini seems smarter and better at architecture and best practices. Claude seems dumber but is more focused on getting things done.
The right solution is going to be a variety of tools and LLMs interacting with each other. But it's going to take real humans having real experience with LLMs to get there. It's not something that you can just dream up on paper and have it work out well since it depends so much on the details of the current models.
from 'product briefs' or something else?
I call this problem the "goldilocks" problem. The task has to be large enough that it outweighs the time necessary to write out a sufficiently detailed specification AND to review and fix the output. It has to be small enough that Claude doesn't get overwhelmed.
The issue with this is, writing a "sufficiently detailed specification" is task dependent. Sometimes a single sentence is enough, other times a paragraph or two, sometimes a couple of pages is necessary. And the "review and fix" phase again is totally dependent and completely unknown. I can usually estimate the spec time but the review and fix phase is a dice roll dependent on the output of the agent.
And the "overwhelming" metric is again not clear. Sometimes Claude Code can crush significant tasks in one shot. Other times it can get stuck or lost. I haven't fully developed an intuition for this yet, how to differentiate these.
What I can say, this is an entirely new skill. It isn't like architecting large systems for human development. It isn't like programming. It is its own thing.
It's management!
I find myself asking very similar questions to you: how much detail is too much? How likely is this to succeed without my assistance? If it does succeed, will I need to refactor? Am I wasting my time delegating or should I just do it?
It's almost identical to when I delegate a task to a junior... only the feedback cycle of "did I guess correctly here" is a lot faster... and unlike a junior, the AI will never get better from the experience.
In my experience, the real "pain" of programming lies in forcing yourself to absorb a flood of information and connecting the dots. Writing code is, in many ways, like taking a walk: you engage in a cognitively light activity that lets ideas shuffle, settle, and mature in the background.
When LLMs write all the code for you, you lose that essential mental rest. The quiet moments where you internalize concepts, spot hidden bugs, and develop a mental map of the system.
I have an emerging workflow orchestrated by Claude Code custom commands and subagents that turns even an informal description of a feature into a full fledged PRD, then an "architect" command researches and produces a well thought out and documented technical design. I can review that design document and then give it to the "planner" command, which breaks it down into Phases and Tasks. Then I have a "developer" command iterate through through and implement the Phases one by one. After each phase it runs a detailed code review using my "review" subagent.
Since I've started using this document-driven, guided workflow I've seen quality of the output noticeably improve.
My pattern with claude code is to let stuff simmer in the background with a detailed PRD, and just turn the screws with progressively more testing and type checking. I'll use repomix to put my entire codebase into gemini 2.5 pro, chat with it for a bit and then ask it to generate a highly detailed work plan for claude code to make the codebase more production hardened/launch ready. If I don't burn my plan tokens first, that gemini prompt can keep claude running for like ~3 hours usually. If you repeat this gemini plan -> claude implement step a few times gemini will eventually start to tell you to stop being a chicken and launch your great app.
My hunch is that good automated testing is an enormous factor with respect to how productive you can get with coding agent tools.
Thorough tests? Just like working without LLMs you can confidently make changes without fear of breaking other parts of the application.
No tests at all? Any change you make is a roll of the dice with respect to how it affects the rest of your existing code.
I do not enjoy spelling out tasks in English and checking that they are done correctly.
Waiting for an AI to complete its task isn't a fun thing at all, and I'd chose the fast 70% correct response any day over the slow 90% correct one. Because by the time the slow one gives you its first attempt, you'd have clarified you need and fixed the output from the fast one.
Sure if we get to the point where the slow system is 100% right, then it's no big deal if it's slow, but we're still far from that point.