I wonder at the end of this if it's the still worth the risk?
A lot of how I form my thoughts is driven by writing code, and seeing it on screen, running into its limitations.
Maybe it's the kind of work I'm doing, or maybe I just suck, but the code to me is a forcing mechanism into ironing out the details, and I don't get that when I'm writing a specification.
I dunno. On the one hand, I keep hearing anecdata, including hackernews comments, friends, and coworkers, suggesting that AI-assisted coding is a literal game changer in terms of productivity, and if you call yourself a professional you'd better damn well lock the fuck in and learn the tools. At the extreme end this takes the form of, you're not a real engineer unless you use AI because real engineering is about using the optimal means to solve problems within time, scale, and budget constraints, and writing code by hand is now objectively suboptimal.
On the other hand, every time the matter is seriously empirically studied, it turns out that overall:
* productivity gains are very modest, if not negative
* there are considerable drawbacks, including most notably the brainrot effect
Furthermore, AI spend is NOT delivering the promised returns to the extent that we are now seeing reversals in the fortunes of AI stocks, up to and including freakin' NVIDIA, as customers cool on what's being offered.
So I'm supposed to be an empiricist about this, and yet I'm supposed to switch on the word of a "cool story bro" about how some guy built an app or added a feature the other day that he totally swears would have taken him weeks otherwise?
I'm like you. I use code as a part of my thought process for how to solve a problem. It's a notation for thought, much like mathematical or musical notation, not just an end product. "Programs must be written for people to read, and only incidentally for machines to execute." I've actually come to love documenting what I intend to do as I do it, esp. in the form of literate programming. It's like context engineering the intelligence I've got upstairs. Helps the old ADHD brain stay locked in on what needs to be done and why. Org-mode has been extremely helpful in general for collecting my scatterbrained thoughts. But when I want to experiment or prove out a new technique, I lean on working directly with code an awful lot.
i go back and forth on this. when i'm working on something where the hard part is the actual algorithm, say custom scheduling logic or a non-trivial state machine, i need my hands in the code because the implementation is the thinking. but for anything where the complexity is in integration rather than logic, wiring up OAuth flows, writing CRUD endpoints, setting up CI pipelines, agents save me hours and the output is usually fine after one review pass. the "code as thought" argument is real but it applies to maybe 20% of what most of us ship day to day. the other 80% is plumbing where the bottleneck is knowing what to build, not how.
> A lot of how I form my thoughts is driven by writing code, and seeing it on screen, running into its limitations.
I completely agree but my thought went to how we are supposed to estimate work just like that. Or worse, planning poker where I'm supposed to estimate work someone else does.
> A lot of how I form my thoughts is driven by writing code, and seeing it on screen, running into its limitations.
Two principles I have held for many years which I believe are relevant both to your sentiment and this thread are reproduced below. Hopefully they help.
First:
When making software, remember that it is a snapshot of
your understanding of the problem. It states to all,
including your future-self, your approach, clarity, and
appropriateness of the solution for the problem at hand.
Choose your statements wisely.
And:
Code answers what it does, how it does it, when it is used,
and who uses it. What it cannot answer is why it exists.
Comments accomplish this. If a developer cannot be bothered
with answering why the code exists, why bother to work with
them?
>but the code to me is a forcing mechanism into ironing out the details, and I don't get that when I'm writing a specification.
This is so on point. The spec as code people try again and again. But reality always punches holes in their spec.
A spec that wasn't exercised in code, is like a drawing of a car, no matter how detailed that drawing is, you can't drive it, and it hides 90% of the complexity.
To me the value of LLMs is not so much in the code they write. They're usually to verbose, start building weird things when you don't constantly micromanage them.
But you can ask very broad questions, iteratively refine the answer, critique what you don't like. They're good as a sounding board.
I liken it to manual versus automated industrial production. I think manual coding will always have its place just like how there are even still people who craft things by manual labor, whether it’s woodworkers only using manual tools or blacksmiths who still manually stoke coke fires that produce very unique and custom products; vs the highly automated production lines we have that produce acceptable forms of something efficiently, and many of them so many people can have them.
I think of it differently: I’ve been coding so long that ironing out the details and working through the specification with AI comes extremely naturally. It’s like how I would talk to a colleague and iterate on their work.
However, the quality of the code produced by LLMs needs to be carefully managed to assure it’s of a high standard. That’s why I formalized a system of checks and balances for my genetic coding that contains architectural guidelines as well as language, specific taste advice.
This is exactly the issue I’m facing especially when working with AI-generated codebases.
Coding is significantly faster but my understanding of the system takes a lot longer because I’m having to merge my mental model with what was produced.
Some people like to lay the brick, some people like to draw the blueprints. I don’t think there is anything wrong with not subscribing to this onslaught on AI tooling, doing the hard work is rewarding. Whether AI will become a standard in how code is written in the future is still to be determined and I think there is a real chance that is where it goes, it shouldn’t hinder your love for doing what you do.
That sounds like the advice of someone who doesn't actually write high-quality code. Perhaps a better title would be "how to get something better than pure slop when letting a chatbot code for you" - and then it's not bad advice I suppose. I would still avoid such code if I can help it at all.
Too bad that software developers are carrying water for those who hate them and mock them for being obsolete in 6-12 months, while they are eating caviar (probably evading sanctions) and clink the champagne glasses in Davos:
The enthusiasm so many devs show for it is also quite bizarre, saying things like "AI makes me so much more productive," with the implication that they will be its primary beneficiaries, and that it won't result in a massive reduction in demand, compensation, and status for developers, adversely affecting them. Even more bizarre when you realize these devs aren't the ones optimizing some popular video codec or writing avionics software for a fighter jet, but instead gluing together NPM packages--probably the first or second rung on on the software "innovator's dilemma" ladder of disruption.
The real value that AI provides is the speed at which it works, and its almost human-like ability to “get it” and reasonably handle ambiguity. Almost like tasking a fellow engineer. That’s the value.
By the time you do everything outlined here you’ve basically recreated waterfall and lost all speed advantage. Might as well write the code yourself and just use AI as first-pass peer review on the code you’ve written.
A lot of the things the writer points out also feel like safeguards against the pitfalls of older models.
I do agree with their 12th point. The smaller your task the easier to verify that the model hasn’t lost the plot. It’s better to go fast with smaller updates that can be validated, and the combination of those small updates gives you your final result. That is still agile without going full “specifications document” waterfall.
Hi i5heu. Given that you seem to use AI tools for generating images and audio versions of your posts, I hope it is not too rude to ask: how much of the post was drafted, written or edited with AI?
The suggestions you make are all sensible but maybe a little bit generic and obvious. Asking ChatGPT to generate advice on effectively writing quality code with AI generates a lot of similar suggestions (albeit less well written).
If this was written with help of AI, I'd personally appreciate a small notice above the blog post. If not, I'd suggest to augment the post with practical examples or anecdotal experience. At the moment, the target group seems to be novice programmers rather than the typical HN reader.
1. Keep things small and review everything AI written, or
2. Keep things bloated and let AI do whatever it wants within the designated interface.
Initially I drew this line for API service / UI components, but it later expanded to other domains. e.g. For my hobby rust project I try to keep "trait"s to be single responsible, never overlap, easy to understand etc etc. but I never look at AI generated "impl"s as long as it passes some sensible tests and conforming the traits.
A good bit of scaffolding and babysitting allows you to let the model run much faster and more efficiently. Building your tool faster. I don't code to code, I code to build something I want.
I'm still writing code. I'm doing it to solve a problem, there's more to writing code than than typing. Recently AI massively simplified "getting started", and all of the tips here are applicable to working well on a team.
My recent experience: I'm porting an app to Mac. It's been in my backlog for ~2 years. With Claude I had a functional prototype in under a day getting the major behavior implemented. I spent the next two weeks refactoring the original app to share as much logic as possible. The first two days was lots of fun. The refactoring was also something I wanted to flush out unit tests, still enjoyable.
The worst part was debugging really bugs introduced to my code from 5 years ago. My functions had naming issues describing the behavior wrong, confusing Claude, that I needed to re-understand to add new features.
Parts of coding are frustrating. Using AI is frustrating for different reasons.
The most frustrating part was rebasing with git to create a sensible history (which I've had to do without AI in the past), reviewing the sheer volume of changes (14k lines) and then deciding "do I want my name on this" which involved cleaning up all the linter warnings I'd self imposed on myself.
Also there is no "compiler" and "type checker" for your SPEC. If you get something wrong in some paragraph somewhere and or contradict something in your spec X paragraphs later - you have to use Mark-1 EyeBall to detect and fix this.
You have just transformed your job from developer to manual spec maintainer - a clerk who has to painstakingly check everything.
> Use strict linting and formatting rules to ensure code quality and consistency. This will help you and your AI to find issues early.
I've always advocated for using a linter and consistent formatting. But now I'm not so sure. What's the point? If nobody is going to bother reading the code anymore I feel like linting does not matter. I think in 10 years a software application will be very obfuscated implementation code with thousands of very solidly documented test cases and, much like compiled code, how the underlying implementation code looks or is organized won't really matter
First article about writing code with AI i can get behind 100%. Stuff i already do, stuff i've thought about doing, and at ideas i've never thought doing ("Mark code review levels" especially is a _great_ idea)
Remember having to write detailed specs before coding? Then folks realized it was faster and easier to skip the specs and write the code? So now are we back to where we were?
One of the problems with writing detailed specs is it means you understand the problem, but often the problem is not understand - but you learn to understand it through coding and testing.
Skip specs, and you often ended up writing the wrong program - at substantial cost.
The main difference now is the parrots have reduced the cost of the wrong program to near zero, thereby eliminating much of the perceived value if a spec.
Every engineering org should be pleading devs to not let AI write tests. They're awful and sometimes they literally don't even assert the code that was generated and instead assert the code in tests.
Every engineering org should be pleading devs to not let AI write code, period. They continue to routinely get stuff wrong and can't be trusted any further than you can throw them.
I found an easier way that Works For Me (TM). I describe the problem to LLM and ask it to solve it step by step, but strictly in the Ask mode, not Agent. Then I copy or even type the linws to the code. If I wouldn't write the line myself, it doesn't go in, and I iterate some more.
I do allow it to write the tests (lots of typing there), but I break them manually to see how they fail. And I do think about what the tests should cover before asking LLM to tell me (it does come up with some great ideas, but it also doesn't cover all the aspects I find important).
Great tool, but it is very easy to be led astray if you are not careful.
I can't help but keep finding it ridiculous how everyone now discovers basic best practices (linting, documentation, small incremental changes) that have been known for ages. It's not needed because of AI, you should have been doing it like this before as well.
Completely disagree. That's like saying that user manuals and driving assistances (e.g. alerts about approaching an object) in cars are only for bad drivers.
You still need to know the hard parts: precisely what you want to build, all domain/business knowledge questions solved, but this tool automates the rest of the coding and documentation and testing.
It's going to be a wild future for software development...
Define data structures manually, ask AI to implement specific state changes. So JSON, C .h or other source files of func sigs and put prompts in there. Never tried the Agents.md monolithic definition file approach
Also I demand it stick to a limited set of processing patterns. Usually dynamic, recursive programming techniques and functions. They just make the most sense to my head and using one style I can spot check faster.
I also demand it avoid making up abstractions and stick to mathematical semantics. Unique namespaces are not relevant to software in the AI era. It's all
about using unique vectors as keys to values.
Stick to one behavior or type/object definition per file.
Only allow dependencies that are designed as libraries to begin with. There is a ton of documentation to implement a Vulkan pipeline so just do that. Don't import an entire engine like libgodot.
And for my own agent framework I added observation of my local system telemetry via common Linux files and commands. This data feeds back in to be used to generate right-sized sched_ext schedules and leverage bpf for event driven responses.
Am currently experimenting with generation of small models of my own data. A single path of images for example not the entire Pictures directory. Each small model is spun akin to a Docker container.
LLMs are monolithic (massive) zip files of the entire web. No one really asking for that. And anyone who needs it already has access to the web itself
so is the 10-20% in velocity worth the money and the process-complexity added? I'm assuming you're measuring your own velocity, not your team's, since that includes time to review and deploy etc.
The first rule is an antipattern. I think describing your architecture or ANY kind of documentation for your AI is an anti-pattern and blows the context window leading to worse results, and actual more deviation.
The controlling systems are not give it more words at the start. Agentic coding needs to work in loop with dedicated context.
You need to think about how can i give as much intent as possible with as little words.
You can built a tremendous amount of custom lint rules ai never needs to read except they miss it.
Every pattern in your repo gets repeated, repo will always win over documentation and when your repo is good structured you don’t need to repeat this to AI
It’s like dev always has been, watch what has gone wrong and make sure the whole type or error can’t happen again.
The post touches very briefly on linting in 7. For me, setting up a large number of static code analysis checks has had the highest impact on code quality.
My hierarchy of static analysis looks like this (hierarchy below is Typescript focused but in principle translatable to other languages):
9. Custom script to ensure shared/util directories are not over stuffed (built this using dependency-cruiser as a library rather than an exec)
10. Security check (semgrep)
I stitch all the above in a single `pnpm check` command and defined an agent rule to run this before marking task as complete.
Finally, I make sure `pnpm check` is run as part of a pre-commit hook to make sure that the agent has indeed addressed all the issues.
This makes a dramatic improvement in code quality to the point where I'm able to jump in and manually modify the code easily when the LLM slot machine gets stuck every now and then.
(Edit: added mention of pre-commit hook which I missed mention of in initial comment)
These kinda things aren’t really the issues I run into. Lack of clarity of thought, overly verbose code, needlessly defensive programming - the stuff that really rots a codebase. Honestly some of the above rules you have I’d want the LLM to ignore at the times if we’re going for maximum maintainability.
this is close to what i've landed on too. the pre-commit hook is non-negotiable. i've had Claude Code report "all checks pass" when there were 14 failing eslint rules. beyond the static analysis though, i keep hitting a harder problem: code that passes every lint rule, compiles clean, and greens the test suite but implements a subtly wrong interpretation of the spec. like an API handler that returns 200 with an empty array instead of 404, technically valid but semantically wrong. evaluating behavioural correctness against intent, not just syntax or type safety, is the gap nobody's really cracked yet. property-based testing helps but it still requires you to formalize the invariants upfront, which is often the hard part.
Not a catch all to fix issues agree with linting. Being very strict with linters has become very cheap with coding agents and it keeps you up to date with code standards and keeps code style homogenous which is very nice when you are reviewing professional code, regardless of who wrote it.
It’s also tricky otherwise if you have to occasionally review lazily written manual code mixed with syntactically formal/clean but functionally incorrect AI code.
80 comments
[ 3.4 ms ] story [ 80.8 ms ] threadA lot of how I form my thoughts is driven by writing code, and seeing it on screen, running into its limitations.
Maybe it's the kind of work I'm doing, or maybe I just suck, but the code to me is a forcing mechanism into ironing out the details, and I don't get that when I'm writing a specification.
On the other hand, every time the matter is seriously empirically studied, it turns out that overall:
* productivity gains are very modest, if not negative
* there are considerable drawbacks, including most notably the brainrot effect
Furthermore, AI spend is NOT delivering the promised returns to the extent that we are now seeing reversals in the fortunes of AI stocks, up to and including freakin' NVIDIA, as customers cool on what's being offered.
So I'm supposed to be an empiricist about this, and yet I'm supposed to switch on the word of a "cool story bro" about how some guy built an app or added a feature the other day that he totally swears would have taken him weeks otherwise?
I'm like you. I use code as a part of my thought process for how to solve a problem. It's a notation for thought, much like mathematical or musical notation, not just an end product. "Programs must be written for people to read, and only incidentally for machines to execute." I've actually come to love documenting what I intend to do as I do it, esp. in the form of literate programming. It's like context engineering the intelligence I've got upstairs. Helps the old ADHD brain stay locked in on what needs to be done and why. Org-mode has been extremely helpful in general for collecting my scatterbrained thoughts. But when I want to experiment or prove out a new technique, I lean on working directly with code an awful lot.
I completely agree but my thought went to how we are supposed to estimate work just like that. Or worse, planning poker where I'm supposed to estimate work someone else does.
Two principles I have held for many years which I believe are relevant both to your sentiment and this thread are reproduced below. Hopefully they help.
First:
And:This is so on point. The spec as code people try again and again. But reality always punches holes in their spec.
A spec that wasn't exercised in code, is like a drawing of a car, no matter how detailed that drawing is, you can't drive it, and it hides 90% of the complexity.
To me the value of LLMs is not so much in the code they write. They're usually to verbose, start building weird things when you don't constantly micromanage them.
But you can ask very broad questions, iteratively refine the answer, critique what you don't like. They're good as a sounding board.
You can check it out here: https://ai-lint.dosaygo.com/
Coding is significantly faster but my understanding of the system takes a lot longer because I’m having to merge my mental model with what was produced.
https://xcancel.com/hamptonism/status/2019434933178306971
And all that after stealing everyone's output.
In reality that man is hoping to IPO in 6-12 months, if anyone is wondering why the “use claude or you’re left behind” is so heavy right now.
By the time you do everything outlined here you’ve basically recreated waterfall and lost all speed advantage. Might as well write the code yourself and just use AI as first-pass peer review on the code you’ve written.
A lot of the things the writer points out also feel like safeguards against the pitfalls of older models.
I do agree with their 12th point. The smaller your task the easier to verify that the model hasn’t lost the plot. It’s better to go fast with smaller updates that can be validated, and the combination of those small updates gives you your final result. That is still agile without going full “specifications document” waterfall.
Next: vibe brain surgery.
/i
The suggestions you make are all sensible but maybe a little bit generic and obvious. Asking ChatGPT to generate advice on effectively writing quality code with AI generates a lot of similar suggestions (albeit less well written).
If this was written with help of AI, I'd personally appreciate a small notice above the blog post. If not, I'd suggest to augment the post with practical examples or anecdotal experience. At the moment, the target group seems to be novice programmers rather than the typical HN reader.
1. Keep things small and review everything AI written, or 2. Keep things bloated and let AI do whatever it wants within the designated interface.
Initially I drew this line for API service / UI components, but it later expanded to other domains. e.g. For my hobby rust project I try to keep "trait"s to be single responsible, never overlap, easy to understand etc etc. but I never look at AI generated "impl"s as long as it passes some sensible tests and conforming the traits.
My recent experience: I'm porting an app to Mac. It's been in my backlog for ~2 years. With Claude I had a functional prototype in under a day getting the major behavior implemented. I spent the next two weeks refactoring the original app to share as much logic as possible. The first two days was lots of fun. The refactoring was also something I wanted to flush out unit tests, still enjoyable.
The worst part was debugging really bugs introduced to my code from 5 years ago. My functions had naming issues describing the behavior wrong, confusing Claude, that I needed to re-understand to add new features.
Parts of coding are frustrating. Using AI is frustrating for different reasons.
The most frustrating part was rebasing with git to create a sensible history (which I've had to do without AI in the past), reviewing the sheer volume of changes (14k lines) and then deciding "do I want my name on this" which involved cleaning up all the linter warnings I'd self imposed on myself.
You have just transformed your job from developer to manual spec maintainer - a clerk who has to painstakingly check everything.
I've always advocated for using a linter and consistent formatting. But now I'm not so sure. What's the point? If nobody is going to bother reading the code anymore I feel like linting does not matter. I think in 10 years a software application will be very obfuscated implementation code with thousands of very solidly documented test cases and, much like compiled code, how the underlying implementation code looks or is organized won't really matter
One of the problems with writing detailed specs is it means you understand the problem, but often the problem is not understand - but you learn to understand it through coding and testing.
So where are we now?
The main difference now is the parrots have reduced the cost of the wrong program to near zero, thereby eliminating much of the perceived value if a spec.
I do allow it to write the tests (lots of typing there), but I break them manually to see how they fail. And I do think about what the tests should cover before asking LLM to tell me (it does come up with some great ideas, but it also doesn't cover all the aspects I find important).
Great tool, but it is very easy to be led astray if you are not careful.
https://github.com/glittercowboy/get-shit-done
You still need to know the hard parts: precisely what you want to build, all domain/business knowledge questions solved, but this tool automates the rest of the coding and documentation and testing.
It's going to be a wild future for software development...
Define data structures manually, ask AI to implement specific state changes. So JSON, C .h or other source files of func sigs and put prompts in there. Never tried the Agents.md monolithic definition file approach
Also I demand it stick to a limited set of processing patterns. Usually dynamic, recursive programming techniques and functions. They just make the most sense to my head and using one style I can spot check faster.
I also demand it avoid making up abstractions and stick to mathematical semantics. Unique namespaces are not relevant to software in the AI era. It's all about using unique vectors as keys to values.
Stick to one behavior or type/object definition per file.
Only allow dependencies that are designed as libraries to begin with. There is a ton of documentation to implement a Vulkan pipeline so just do that. Don't import an entire engine like libgodot.
And for my own agent framework I added observation of my local system telemetry via common Linux files and commands. This data feeds back in to be used to generate right-sized sched_ext schedules and leverage bpf for event driven responses.
Am currently experimenting with generation of small models of my own data. A single path of images for example not the entire Pictures directory. Each small model is spun akin to a Docker container.
LLMs are monolithic (massive) zip files of the entire web. No one really asking for that. And anyone who needs it already has access to the web itself
1. Have the LLM write code based on a clear prompt with limited scope 2. Look at the diff and fix everything it got wrong
That's it. I don't gain a lot in velocity, maybe 10-20%, but I've seen the code, and I know it's good.
The controlling systems are not give it more words at the start. Agentic coding needs to work in loop with dedicated context.
You need to think about how can i give as much intent as possible with as little words.
You can built a tremendous amount of custom lint rules ai never needs to read except they miss it.
Every pattern in your repo gets repeated, repo will always win over documentation and when your repo is good structured you don’t need to repeat this to AI
It’s like dev always has been, watch what has gone wrong and make sure the whole type or error can’t happen again.
My hierarchy of static analysis looks like this (hierarchy below is Typescript focused but in principle translatable to other languages):
1. Typesafe compiler (tsc)
2. Basic lint rules (eslint)
3. Cyclomatic complexity rules (eslint, sonarjs)
4. Max line length enforcement (via eslint)
5. Max file length enforcement (via eslint)
6. Unused code/export analyser (knip)
7. Code duplication analyser (jscpd)
8. Modularisation enforcement (dependency-cruiser)
9. Custom script to ensure shared/util directories are not over stuffed (built this using dependency-cruiser as a library rather than an exec)
10. Security check (semgrep)
I stitch all the above in a single `pnpm check` command and defined an agent rule to run this before marking task as complete.
Finally, I make sure `pnpm check` is run as part of a pre-commit hook to make sure that the agent has indeed addressed all the issues.
This makes a dramatic improvement in code quality to the point where I'm able to jump in and manually modify the code easily when the LLM slot machine gets stuck every now and then.
(Edit: added mention of pre-commit hook which I missed mention of in initial comment)
For the pre-commit hook, I assume you run it on just the files changed?
> Custom script to ensure shared/util directories are not over stuffed (built this using dependency-cruiser as a library rather than an exec)
Would you share this?
BUT, what is the point of max line length enforcement, just to see if there are crazy ternary operators going on?
It’s also tricky otherwise if you have to occasionally review lazily written manual code mixed with syntactically formal/clean but functionally incorrect AI code.