Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.
Agreed with Reading! This alone isn't enough though. PreLLM too it wasn't just reading code to review. Someone did the hard work or crafting the code and each unit test would tell you the weird corner cases to deal with and factor that into changing your code. One person owned a part of the codebase and was an expert.
Not to mention reading isnt easy when the velocity of code pumping in is 3-5x more. Its exhausting and reading becomes skimming.
Conversely I'm a solo-dev and I ship a lot of slop I don't even look at. Granted my work is just basic CRUD apps, and I focus my efforts on validating important consequences (like does this break accounting invariants or something)
Yeah, but in the corporate world now, you're being judged by how fast you get shit checked in. So, if you need to understand every line these days... you're going to end up in the bottom 25% of the company pretty quickly. Not saying you're wrong, just saying how fucked things are becoming.
While the tips are good to handle the volume, I still think this sets code owner on a dangerous path.
AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high.
The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.
Something funny happens when you reject code. The agent realises it is garbage.
I'm personally of the opinion that the tools for reviewing LLM generated code are awful. 99% of the time I want to do line by line comments and tell it everything it did wrong. Given that information, the next iteration would be much more up to my standards.
The same also applies to other people's LLM generated code. Yeah sure they can just pass on the comments to the LLM, but that will just mean more iterations and them losing their job.
Understanding has always been the bottleneck. That's why LLMs aren't actually helpful: they speed up the part which is easy (typing characters into your editor), but are neutral or even harmful on the part which is hard (understanding the problem and how best to solve it).
The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.
Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.
(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)
LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
LLMs usually points to the most idiotic future trajectory on my work, and I have to curse it inorder to let it keep up with my refined understanding.
But what else would one expect from a probabilistic weighted next token predictor, other than to conduct probabilistic search which are 99.99% deadends.
But LLMs can pave the way towards constructing resilient and correct architecture which can be iterated fast by a human.
Architecture and determinism is where my money is in.
I've been using Geoffrey's /explain-diff skill in my replace-github-with-tailor-fit-personal-software journey, and I'm liking it. I recommend at least giving it a try.
Understanding was always the bottleneck. The way LLMs speed up your work is by letting you get code without taking the time to understand it. If you want to understand your code, LLMs are a net loss.
If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
This is a temporary bottleneck. AI is moving so fast that this will change. Wait six months and this article is no longer relevant.
About a year ago most people were still typing code. Having an agent do ALL code was crazy.
Within a year or two years at most, a lot of people will stop trying to understand code. The onus will shift to testing and QAing.
I know this is hard to hear but that’s the trendline. That’s where all of this is converging. Everyone’s to busy trying to lock themselves down as an expert of the new “paradigm” but it’s all moving so fast that the paradigm now won’t be the paradigm of tomorrow.
Understanding has always been the bottleneck. Sometimes AI helps with it like explaining things pretty well with diagrams. However, in general I agree that more code is being generated per developer and it's difficult to keep up with the phase of new changes and understand it.
Hot take: I look at the level of abstraction that matters most to me. When I encounter cognitive debt (usually due to sleepy sessions where I’m mostly “encouraging” Claude), I ask it to step back to clarify the overall purpose. If I get really stuck, I have it visualize the processes involved. Usually, the hard part is giving specific enough feedback to get a specific enough response within a much broader set of working material.
"So I asked Claude to make me a video game — a command center where I do the port myself, step by step, watching the visible effects and the file tree evolve. It produced a UI where I click buttons to run the port step by step, with my old site and new site running side by side."
It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it.
Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.
Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code.
The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed.
This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.
My personal view is that programming languages are amazing tools for understanding. Some more than others, but even the worst—the most verbose, the lowest level—are better than they have any right to be.
So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.
Thanks for the post Geoffrey. I have been thinking about this a bit and wanted to come to these sorts of conclusions, you have saved me a lot of work (lol I have no ego that I have to figure it out I am happy you did).
Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?
How does everyone feel about the “don’t read the code” stuff that folks are saying? I certainly do not support it but I’m curious to hear what other folks thoughts are
66 comments
[ 0.21 ms ] story [ 30.9 ms ] threadGreat code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.
Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!
AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high.
The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.
Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
I'm personally of the opinion that the tools for reviewing LLM generated code are awful. 99% of the time I want to do line by line comments and tell it everything it did wrong. Given that information, the next iteration would be much more up to my standards.
The same also applies to other people's LLM generated code. Yeah sure they can just pass on the comments to the LLM, but that will just mean more iterations and them losing their job.
The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.
Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.
(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)
LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
LLMs usually points to the most idiotic future trajectory on my work, and I have to curse it inorder to let it keep up with my refined understanding.
But what else would one expect from a probabilistic weighted next token predictor, other than to conduct probabilistic search which are 99.99% deadends.
But LLMs can pave the way towards constructing resilient and correct architecture which can be iterated fast by a human.
Architecture and determinism is where my money is in.
If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
About a year ago most people were still typing code. Having an agent do ALL code was crazy.
Within a year or two years at most, a lot of people will stop trying to understand code. The onus will shift to testing and QAing.
I know this is hard to hear but that’s the trendline. That’s where all of this is converging. Everyone’s to busy trying to lock themselves down as an expert of the new “paradigm” but it’s all moving so fast that the paradigm now won’t be the paradigm of tomorrow.
It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it.
Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.
The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed.
This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.
So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.
Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?
I guess a wall of text is the way now