I used to joke about prompt engineering. But by jiminy it is a thing now. I swear sometimes I waste a good 10-20minutes writing up a good prompt and initial plan just so that claudecode can systematically implement something.
My usage is nearly the same as OP. Plan plan plan save as a file and then new context and let it rip.
That's the one thing I'd love, a good cli (currently using charm and cc) which allows me to have an implementation model, a plan model and (possibly) a model per sub agent. Mainly so I can save money by using local models for implementation and online for plans or generation or even swapping back. Charm has been the closest I've used so far allowing me to swap back and forth and not lose context. But the parallel sub-agent feature is probably one of the best things claudecode has.
(Yes I'm aware of CCR, but could never get it to use more than the default model so :shrug:)
I’ve recently tried out Claude Code for a bit, I’ll make sure to give the suggested approach a go! It sounds like a nice workflow.
But I’m negatively surprised with the amount of money CC costs. Just a simple refactoring cost me about 5min + 15min review and 4usd, had I done it myself it might have taken 15-20min as well.
How much money do you typically spend on features using CC? Nobody seems to mention this
Over the last 2 weeks (evenings only) I've spend a lot of time crafting the "perfect prompt" for claude code to one shot the project. I ended up with a rather small CLAUDE.md file that references 8 other MD files, ranging from project_architecture, models_spec, build_sequence, test_hierarchy, test_scenarios, and some other files.
It is a project for model based governance of Databricks Unity Catalog, with which I do have quite a bit of experience, but none of the tooling feels flexible enough.
Eventually I ended up with 3 different subagents that supported in the development of the actual planning files; a Databricks expert, a Pydantic expert, and a prompt expert.
The improvement on the markdown files was rather significant with the aid of these. Ranging from old pydantic versions and inconsistencies, to me having some misconceptions about unity catalog as well.
Yesterday eve I gave it a run and it ran for about 2 hours with me only approving some tool usage, and after that most of the tools + tests were done.
This approach is so different than I how used to do it, but I really do see a future in detailed technical writing and ensuring we're all on the same page.
In a way I found it more productive than going into the code itself.
A downside I found is that with code reading and working on it I really zone in.
With a bunch of markdown docs I find it harder to stay focused.
It’s interesting to me that trying to optimise AI tools is leading many engineers to discover the value in good communication and expectation setting. The diva/autist stereotype of 10x programmers is due for a review.
I too recently discovered this workflow and I'm blown by it. The key IMHO is first to give claude as low requirements as possible and let it's plan mode roam freely. Writing a reporting for sales metrics? "Ultrathink relevant sales metrics" and it will give you a lot to start ranking which you want, maybe add some that are missing. Then create a new directory for this feature and ask it to write the plan to a file. Then proceed to create an implementation plan, ask it to find all the relevant data from the database and write how to query it. Then finally let it implement it and write tests and end user documentation. And send it to QA.
Need sales forecasting? This used to be an enterprise feature that 10 years ago would have needed a large team to implement correctly. Claude implements a docker container in one afternoon.
It really changes how I see software now. Before there were NDAs and intellectual property and companies too great care not to leak their source code.
Now things have changed, have a complex ERP system that took 20 years to develop? Well, claude can re-implement it in a flash. And write documentation and tests for it. Maybe it doesn't work quite that well yet, but things are moving fast.
I have been using exactly author's approach with "great success: (quote Borat) over the last two months. The first month with CC was also mainly nudging it along and that only gets you so far.
But since then I have come to have it always write ARCHITECTURE.md and IMPLEMENTATION.md when doing a new feature and CLAUDE-CONTINUE.md. All three live in the resp. folder of the feature (in my case, it's often a new crate or a new module, as I write Rust).
The architecture one is usually the result of some back and forth with CC much like the author describes it. Once that is nailed it writes that out and also the implementation. These are not static ofc, they may get updated during the process but the longer you spend discussing with CC and thinking about what you're doing the less likely this is necessary. Really no surprise there -- this works the same way in meat space. :]
I have an instruction in CLAUDE.md that it should update CLAUDE-CONTINUE.md with the current status, referencing both the other documents, when the context is 2% away from getting compacted.
After the compact it reads the resp. CLAUDE-CONTINUE.md (automatically, since it's referenced in CLAUDE.md) and then usually continues as if nothing happened. Without this my mileage varied as it needs to often read a lot of code (again) first and calibrated to what parts of architecture and implementation it did, before the compact.
I often also have it write out stuff that is needed in dependencies that I maintain or that are part of the project so then it creates ARCHITECTURE-<feature>-<crate>.md and I just copy that over to the resp. repo and tell another CC instance there to write the implementation document and send it off.
A lot of stuff I do is done via Terry [1] and this approach has worked a treat for me. Shout out to these guys, they rock.
Edit: P.S. I have 30+ years R&D experience in my field so I have deep understanding of what I do (computer graphics, system programming, mostly). I have quite a few friends with a decade or less of R&D experience and they struggle to get the same amount of shit done with CC or Ai.
The models are not there yet, you need the experience. I also mainly formulate concisely what I want and what the API should look and the go back and forth with CC, not start with a fuzzy few sentences and cross my fingers that what it comes up with is something I may like and can then mold a tad.
I also found that not getting weird bugs that the model may chase for several "loops" seem correlated with the amount of statically-typed code. I.e. I've been recently working on a Python code base that interfaces with Rust and the number of times CC shot itself in the foot because it assumed a foo was a [foo] and stuff like that is just astounding. This obviously doesn't happen in Rust, the language/compiler catches it and the model 'knows' it can't get away with it so it seems to exercises more rigor (but I may be 'hallucinating' that).
TLDR; I came to the conclusion that statically-typed languages get you higher returns with these models for this reason.
This has been my experience with replit as well. It needs to use design docs as the source of task and truth, as it starts to crumble as the app size increases.
With OpenAI I find ChatGPT just slows to a crawl and the chat becomes unresponsive. Asking it to make a document, to import into a new chat, helps with that.
On a human level, it makes me think that we should do the same ourselves. Reflect, document and dump our ‘memory’ into a working design doc. To free up ourselves, as well as our LLMs.
Does any one know "roughly" how ClaudeCode compares costwise these days to Cursor using OpenAI api? I just remember it being well so expensive I ended up paying hundreds of dollars for it a month
What I don't get about all the "if you plan it out first, it gets better" approach is, how did they work before?!
For anything bigger than small size features, I always think about what I do and why I do things. Sometimes in my head, sometimes on paper, a Confluence page or a white board.
I don't really get it. 80 % of software engineering is to figure out what you need and how to achieve this. You check with the stake holders, write down the idea of what you want to do and WHY you want to do it. You do some research.
Last 20 % of the process is coding.
This was always the process. You don't need AI for proper planning and defining your goals.
interesting. that living plan document is something humans learn to make and update themselves. these problems are dynamical, requiring the solver to maintain state, and the plan is what records that.
doing it for the LLM really highlights that limitation. they arent trained statefully, not at the foundation model, where it matters. that state gets reproduced on top of the model in the form of "reasoning" and "chain of thought" but that level of scaffolding is a classic example of the bitter lesson. like semantic trees of old.
the representation learning + transformer model needs to be evolved to handle state, then it should be able to do these things itself
This is the key to getting decent feature work out of Claude Code. I've had good success recently using GPT-5 High (in Cursor) to write the plan, then take that to Claude Code to implement.
You can get an extra 15-20% out of it if you also document the parts of the codebase you expect to change first. Let the plan model document how it works, architecture and patterns. Then plan your feature with this in the context. You'll get better code out of it.
Also, make sure you review, revise and/or hand edit the docs and plans too. That pays significant dividends down the line.
Has anyone figured out an elegant way to add front-end design to a process like this? Every implementation I see people use includes either vague references to front-end frameworks, or figma images. It doesn't feel like a cohesive design solution.
I have a folder of scss files containing the utility classes and custom properties that make up the design. By instructing it to use those, and to reference similar files, it more or less conforms nicely to the existing design language.
That is another post praising the waterfall model. What Claude Photocopier does here is steal from hundreds of similar projects. It does not design anything and neither are you.
Does anyone know how to get support from Anthropic? I purchased the Claude.ai Pro sub to try it out and disabled auto-renewal, then tried to remove my payment method from my account, just in case.
You can't.
So I tried to reach support. There's no email, no phone number, just a THIRD-PARTY AI chatbot.
Well guess what, the Send Message button in the text field is disabled.
This is infuriating and puts me off the whole product and maybe I'll just file a chargeback.
This seems similar to the way I use Visual Studio Code/ChatGPT5(preview) {I think it's being payed for by my github copilot subscription, but I'm not actually sure these days}
Having tried non-agent LLMs for code, things tend to break and quickly devolve. The agent mode of working with LLMs to build code is a step change improvement for me. I'm not a python programmer, but have been working on a pile of new code that it's built for me, and I'm fairly impressed at what's been achieved in the past week.
Once I get done, and can run a small LLM in my emulated BitGrid, then I'll back off and try to grok the code. It's been a series of small exploratory steps, with a few corrections on my part to keep the overall design going where I want. I'm much more hopeful about the future of "LLM as programming buddy", now that I've actually used an agent like this.
Does anyone else here use the Visual Studio Code/ChatGPT5 combo?
23 comments
[ 1243 ms ] story [ 1679 ms ] threadMy usage is nearly the same as OP. Plan plan plan save as a file and then new context and let it rip.
That's the one thing I'd love, a good cli (currently using charm and cc) which allows me to have an implementation model, a plan model and (possibly) a model per sub agent. Mainly so I can save money by using local models for implementation and online for plans or generation or even swapping back. Charm has been the closest I've used so far allowing me to swap back and forth and not lose context. But the parallel sub-agent feature is probably one of the best things claudecode has.
(Yes I'm aware of CCR, but could never get it to use more than the default model so :shrug:)
But I’m negatively surprised with the amount of money CC costs. Just a simple refactoring cost me about 5min + 15min review and 4usd, had I done it myself it might have taken 15-20min as well.
How much money do you typically spend on features using CC? Nobody seems to mention this
It is a project for model based governance of Databricks Unity Catalog, with which I do have quite a bit of experience, but none of the tooling feels flexible enough.
Eventually I ended up with 3 different subagents that supported in the development of the actual planning files; a Databricks expert, a Pydantic expert, and a prompt expert.
The improvement on the markdown files was rather significant with the aid of these. Ranging from old pydantic versions and inconsistencies, to me having some misconceptions about unity catalog as well.
Yesterday eve I gave it a run and it ran for about 2 hours with me only approving some tool usage, and after that most of the tools + tests were done.
This approach is so different than I how used to do it, but I really do see a future in detailed technical writing and ensuring we're all on the same page. In a way I found it more productive than going into the code itself. A downside I found is that with code reading and working on it I really zone in. With a bunch of markdown docs I find it harder to stay focused.
Curious times!
Assumptions without evaluation are not trustworthy.
Need sales forecasting? This used to be an enterprise feature that 10 years ago would have needed a large team to implement correctly. Claude implements a docker container in one afternoon.
It really changes how I see software now. Before there were NDAs and intellectual property and companies too great care not to leak their source code.
Now things have changed, have a complex ERP system that took 20 years to develop? Well, claude can re-implement it in a flash. And write documentation and tests for it. Maybe it doesn't work quite that well yet, but things are moving fast.
But since then I have come to have it always write ARCHITECTURE.md and IMPLEMENTATION.md when doing a new feature and CLAUDE-CONTINUE.md. All three live in the resp. folder of the feature (in my case, it's often a new crate or a new module, as I write Rust).
The architecture one is usually the result of some back and forth with CC much like the author describes it. Once that is nailed it writes that out and also the implementation. These are not static ofc, they may get updated during the process but the longer you spend discussing with CC and thinking about what you're doing the less likely this is necessary. Really no surprise there -- this works the same way in meat space. :]
I have an instruction in CLAUDE.md that it should update CLAUDE-CONTINUE.md with the current status, referencing both the other documents, when the context is 2% away from getting compacted.
After the compact it reads the resp. CLAUDE-CONTINUE.md (automatically, since it's referenced in CLAUDE.md) and then usually continues as if nothing happened. Without this my mileage varied as it needs to often read a lot of code (again) first and calibrated to what parts of architecture and implementation it did, before the compact.
I often also have it write out stuff that is needed in dependencies that I maintain or that are part of the project so then it creates ARCHITECTURE-<feature>-<crate>.md and I just copy that over to the resp. repo and tell another CC instance there to write the implementation document and send it off.
A lot of stuff I do is done via Terry [1] and this approach has worked a treat for me. Shout out to these guys, they rock.
Edit: P.S. I have 30+ years R&D experience in my field so I have deep understanding of what I do (computer graphics, system programming, mostly). I have quite a few friends with a decade or less of R&D experience and they struggle to get the same amount of shit done with CC or Ai.
The models are not there yet, you need the experience. I also mainly formulate concisely what I want and what the API should look and the go back and forth with CC, not start with a fuzzy few sentences and cross my fingers that what it comes up with is something I may like and can then mold a tad.
I also found that not getting weird bugs that the model may chase for several "loops" seem correlated with the amount of statically-typed code. I.e. I've been recently working on a Python code base that interfaces with Rust and the number of times CC shot itself in the foot because it assumed a foo was a [foo] and stuff like that is just astounding. This obviously doesn't happen in Rust, the language/compiler catches it and the model 'knows' it can't get away with it so it seems to exercises more rigor (but I may be 'hallucinating' that).
TLDR; I came to the conclusion that statically-typed languages get you higher returns with these models for this reason.
[1] https://www.terragonlabs.com/
With OpenAI I find ChatGPT just slows to a crawl and the chat becomes unresponsive. Asking it to make a document, to import into a new chat, helps with that.
On a human level, it makes me think that we should do the same ourselves. Reflect, document and dump our ‘memory’ into a working design doc. To free up ourselves, as well as our LLMs.
For anything bigger than small size features, I always think about what I do and why I do things. Sometimes in my head, sometimes on paper, a Confluence page or a white board.
I don't really get it. 80 % of software engineering is to figure out what you need and how to achieve this. You check with the stake holders, write down the idea of what you want to do and WHY you want to do it. You do some research.
Last 20 % of the process is coding.
This was always the process. You don't need AI for proper planning and defining your goals.
doing it for the LLM really highlights that limitation. they arent trained statefully, not at the foundation model, where it matters. that state gets reproduced on top of the model in the form of "reasoning" and "chain of thought" but that level of scaffolding is a classic example of the bitter lesson. like semantic trees of old.
the representation learning + transformer model needs to be evolved to handle state, then it should be able to do these things itself
You can get an extra 15-20% out of it if you also document the parts of the codebase you expect to change first. Let the plan model document how it works, architecture and patterns. Then plan your feature with this in the context. You'll get better code out of it.
Also, make sure you review, revise and/or hand edit the docs and plans too. That pays significant dividends down the line.
You can't.
So I tried to reach support. There's no email, no phone number, just a THIRD-PARTY AI chatbot.
Well guess what, the Send Message button in the text field is disabled.
This is infuriating and puts me off the whole product and maybe I'll just file a chargeback.
It becomes a collaborative design partner.”
You’re completely right!
Having tried non-agent LLMs for code, things tend to break and quickly devolve. The agent mode of working with LLMs to build code is a step change improvement for me. I'm not a python programmer, but have been working on a pile of new code that it's built for me, and I'm fairly impressed at what's been achieved in the past week.
Once I get done, and can run a small LLM in my emulated BitGrid, then I'll back off and try to grok the code. It's been a series of small exploratory steps, with a few corrections on my part to keep the overall design going where I want. I'm much more hopeful about the future of "LLM as programming buddy", now that I've actually used an agent like this.
Does anyone else here use the Visual Studio Code/ChatGPT5 combo?