This may produce some successes, but it's so much more work than just writing the code yourself that it's pointless. This structured way of working with generative AI is so strict that there is no scaling it up either. It feels like years since this was established to be a waste of time.
If the goal is to start writing code not knowing much, it may be a good way to learn how and establish a similar discipline within yourself to tackle projects? I think there's been research that training wheels don't work either though. Whatever works and gets people learning to write code for real can't be bad, right?
The most important thing is to have a strong plan cycle in front of you agent work, if you do that, agents are very reliable. You need to have a deep research cycle that basically collects a covering set of code that might need to be modified for a feature, feeds it into gemini/gpt5 to get a broad codebase level understanding, then has a debate cycle on how to address it, with the final artifact being a hyper detailed plan that goes file by file and provides an outline of changes required.
Beyond this, you need to maintain good test coverage, and you need to have agents red-team your tests aggressively to make sure they're robust.
If you implement these two steps your agent performance will skyrocket. The planning phase will produce plans that claude can iterate on for 3+ hours in some cases, if you tell it to complete the entire task in one shot, and the robust test validation / change set analysis will catch agents solving an easier problem because they got frustrated or not following directions.
> The planning phase will produce plans that claude can iterate on for 3+ hours in some cases, if you tell it to complete the entire task in one shot, and the robust test validation / change set analysis will catch agents solving an easier problem because they got frustrated or not following directions.
Don't you run into context nightmares though? I was coming up with very detailed plans (using zen to vet with other models), but I found claude just doing the wrong thing a lot of the time, ignoring and / or forgetting very specific instructions and rules, especially across context compactions.
There's this one time that really sticks out in my mind because I had to constantly correct it; when to use ->> versus -> and handle null / type checks with PostgreSQL JSONB. Vibe coders would miss this sort of thing with testing unless they knew that JSONB null is not the same as SQL NULL (and other types too). When working with nested data, you probably won't have test coverage for it. This is just one of many examples too.
Even before you plan, you need to feed it enough relevant context, to make sure the plan is not based on hallucinations it assumes about the system. The best approach I found:
1. Make it read relevant pieces of code and explain it to me
2. Explain my problem and ask it to come up with a plan, iterate if needed
3. Allow it to execute the plan, watch it do it, interrupt and correct when needed
4. Have it do code review using a sub agent, focusing on correctness, avoiding leftover code, etc
There's some irony; far from handling the details, LLMs are forcing programmers to adopt hyper-detailed, disciplined practices. They've finally cajoled software developers into writing documentation! Worth noting we've always had the capacity to implement these practices to improve HUMAN collaboration, but rarely bothered.
What i keep seeing missing for AI-labor replacement discussions is that technology may seem to replace human labor, but it doesn't really replace human accountability.
Organizations many times seem capable to diffuse blame for mistakes within their human beaurocracy but as beaurocracy is reduced with AI, individuals become more exposed.
This alone - in my view - is sufficient counterpressure to fully replace humans in organizations.
Shorter reply: if my AI setup fails I'm the one to blame. If I do a bad job at helping coworkers perform better is the blame fully mine?
My experience writing in a professional setting is that people mostly don't read what I write, and the more effort I put into being thorough the less likely that it will be read.
Reminds me when I was demonstrating Claude code to a friend recently. My friend was a huge cursor user and was just curious about the cli tool and stuff.
In the end, regardless of framework or approach, I believe there is a way to go about using llms that will optimize work for developers. I worked with this tech lead who reviews all PRs and insists on imports arranged in a specific order. I found it insulting but did it anyway. Now I don’t - the not does.
The same way that llms can be really helpful in planning and building out specific things like rest endpoints, small web components, single functions or classes and so on.
Glad people are attempting to work on such potential solution for approaching work to take advantage of these new tools
Can anyone help me on how to integrate this with Claude-Code?
I went through it, I already follow few things manually but when I think of integrating most of the parts (not all), I don't know where should I put it for the Coding-LLM to understand. I fear if I put everything in Claude.md it will be just too much context for the CC.
Back in the day, when business computing emerged (COBOL, Mainframes...), it appear the distinction between systems analysts and programmers. Analyst understand business needs, programmers implemented those specs in code.
Years later, the industry evolved to integrate both roles, and new methodologies and new roles appear.
Now humans write specs, and AI agents write code. Role separation is a principle of labor division since Plato.
16 comments
[ 2.6 ms ] story [ 43.2 ms ] threadIf the goal is to start writing code not knowing much, it may be a good way to learn how and establish a similar discipline within yourself to tackle projects? I think there's been research that training wheels don't work either though. Whatever works and gets people learning to write code for real can't be bad, right?
Beyond this, you need to maintain good test coverage, and you need to have agents red-team your tests aggressively to make sure they're robust.
If you implement these two steps your agent performance will skyrocket. The planning phase will produce plans that claude can iterate on for 3+ hours in some cases, if you tell it to complete the entire task in one shot, and the robust test validation / change set analysis will catch agents solving an easier problem because they got frustrated or not following directions.
Don't you run into context nightmares though? I was coming up with very detailed plans (using zen to vet with other models), but I found claude just doing the wrong thing a lot of the time, ignoring and / or forgetting very specific instructions and rules, especially across context compactions.
There's this one time that really sticks out in my mind because I had to constantly correct it; when to use ->> versus -> and handle null / type checks with PostgreSQL JSONB. Vibe coders would miss this sort of thing with testing unless they knew that JSONB null is not the same as SQL NULL (and other types too). When working with nested data, you probably won't have test coverage for it. This is just one of many examples too.
1. Plan, review the plan.
2. Review the code during changes before even it finish and fix ASAP you see drift.
3. Then again review
4. Add tests & use all quality tools don't rely 100% on LLM.
5. Don't trust LLM reviews for own produced code as it's very biased.
This is basic steps that you do as you like.
Avoid FULL AUTOMATED AGENT pipeline where you review the code only at the end unless it's very small task.
1. Make it read relevant pieces of code and explain it to me
2. Explain my problem and ask it to come up with a plan, iterate if needed
3. Allow it to execute the plan, watch it do it, interrupt and correct when needed
4. Have it do code review using a sub agent, focusing on correctness, avoiding leftover code, etc
5. Then I review it myself
Organizations many times seem capable to diffuse blame for mistakes within their human beaurocracy but as beaurocracy is reduced with AI, individuals become more exposed.
This alone - in my view - is sufficient counterpressure to fully replace humans in organizations.
Shorter reply: if my AI setup fails I'm the one to blame. If I do a bad job at helping coworkers perform better is the blame fully mine?
It turns out that writing and maintaining documentation is just that universally hated.
In the end, regardless of framework or approach, I believe there is a way to go about using llms that will optimize work for developers. I worked with this tech lead who reviews all PRs and insists on imports arranged in a specific order. I found it insulting but did it anyway. Now I don’t - the not does.
The same way that llms can be really helpful in planning and building out specific things like rest endpoints, small web components, single functions or classes and so on.
Glad people are attempting to work on such potential solution for approaching work to take advantage of these new tools
Years later, the industry evolved to integrate both roles, and new methodologies and new roles appear.
Now humans write specs, and AI agents write code. Role separation is a principle of labor division since Plato.