Comprehension debt: A ticking time bomb of LLM-generated code (codemanship.wordpress.com)

This is not just for LLM code. This is for any code that is written by anyone except yourself. A new engineer at Google, for example, cannot hit the ground running and make significant changes to the Google algorithm without months of "comprehension debt" to pay off.

However, code that is well-designed by humans tends to be easier to understand than LLM spaghetti.

Shouldn't you be getting the LLM to also generate test cases to drive the code and also enforce coding standards on the LLM to generate small easily comprehensible software modułes with high quality inline documentation.

Is this something people are doing?

A softer version of this has existed since word processing and Xerox machines (copiers) took off, in law and regulations. Tax code, zoning code etc exploded in complexity once words became immensely easy to create and copy.

I'm sure future LLMs will be able to comprehend more. So the debt, similarly to real world debt, is fine, as long as the line goes up.

I think this is a relative succinct summary of the downside case for LLM code generation. I hear a lot of this and as someone who enjoys a well-structured codebase, I have a lot of instinctive sympathy.

However I think we should be thinking harder about how coding will change as LLMs change the economics of writing code: - If the cost of delivering a feature is ~0, what's the point in spending weeks prioritizing it? Maybe Product becomes more like an iterative QA function? - What are the risks that we currently manage through good software engineering practices and what's the actual impact of those risks materializing? For instance, if we expose customer data that's probably pretty existential, but most companies can tolerate a little unplanned downtime (even if they don't enjoy it!). As the economics change, how sustainable is the current cost/benefit equilibrium of high-quality code?

We might not like it but my guess is that in ≤ 5 years actual code is more akin to assembler where sure we might jump in and optimize but we are really just monitoring the test suites and coverage and risks rather than tuning whether or not the same library function is being evolved in a way which gives leverage across the code base.

This was a pre-existing problem, even if reliance on LLMs is making it worse.

Naur (https://gwern.net/doc/cs/algorithm/1985-naur.pdf) called it "theory building":

> The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Revival of a program is the rebuilding of its theory by a new programmer team.

Lamport calls it "programming ≠ coding", where programming is "what you want to achieve and how" and coding is telling the computer how to do it.

I strongly agree with all of this. Even if your dev team skipped any kind of theory-building or modelling phase, they'd still passively absorb some of the model while typing the code into the computer. I think that it's this last resort of incidental model building that the LLM replaces.

I suspect that there is a strong correlation between programmers who don't think that there needs to be a model/theory, and those who are reporting that LLMs are speeding them up.

My experience is that LLM too often finds solutions that work, but are way more complex than necessary. It is easiest to recognize and remove such complexity when the code is originally created, because at this time the author should have the best understanding of the problem being solved, but this requires extra time and effort. Once the overly complex code is committed, it is much harder to recognize the complexity is not needed. Readers/maintainers of code usually assume that the existing code solves real world problem, they do not have enough context to recognize that much simpler solution could work as well.

So far I have found two decent uses for LLM generated code.

First, refactoring code. Specifically, recently I used it on a library that had solid automated testing coverage. I needed to change the calling conventions of a bunch of methods and classes in the library, but didn’t want to rewrite the 100+ unit tests by hand. Claude did this quickly and without fuss.

Second is one time use code. Basically let’s say you need to convert a bunch of random CVS files to a single YAML file, or convert a bunch of video files in different formats to a single standard format, or find any photos in your library that are out of focus. This works reasonably well.

Bonus one is just generating sample code for well known libraries.

I have been curious what would happen if I handed something like Claude a whole server and told it to manage it however it wants with relatively little instruction.

I'm not of the "LLMs will replace all software developers within a year" mindset, but this critique feels a bit overstated.

The challenge of navigating rapidly changing or poorly documented code isn’t new: It’s been a constant at every company I’ve worked with. At larger organizations the sheer volume of code, often written by adjacent teams, will outpace your ability to fully understand it. Smaller companies tend to iterate so quickly (and experience so much turnover) that code written two weeks ago might already be unrecognizable, if the original author is even still around after those two weeks!

The old adage still applies: the ability to read code is more crucial than the ability to write it. LLMs just amplify that dynamic. The only real difference is that you should assume the author is gone the moment the code lands. The author is ephemeral, or they went on PTO/quit immediately afterward: Whatever makes you more comfortable.

So many of these concepts only make sense under the assumption that AI will not get better and humans will continue to pour over code by hand.

They won't. In a year or two these will be articles that get linked back to similar to "Is the internet just a fad?" articles of the late 90s.

> ... taking developers to modify or fix code

Fix your tests not your resulting code

You learn more when you take notes. In the same way, I understand the structure of the code better when my hands are on keyboard.

I like writing code because eventually I have to fix code. The writing will help me have a sense for what's going on. Even if it will only be 1% of the time I need to fix a bug, having that context is extremely valuable.

Then reserve AI coding when there's true boilerplate or near copy-paste of a pattern.

I haven't used them in big codebases, but they were also able to help me understand the code they generated. Isn't this feasible (yet) on big codebases?

LLMs absolutely produce reams of hard-to-debug code. It's a real problem.

But "Teams that care about quality will take the time to review and understand LLM-generated code" is already failing. Sounds nice to say, but you can't review code being generated faster than you can read it. You either become a bottleneck (defeats the point) or you rubber-stamp it (creates the debt). Pick your poison.

Everyone's trying to bolt review processes onto this. That's the wrong layer. That's how you'd coach a junior dev, who learns. AI doesn't learn. You'll be arguing about the same 7 issues forever.

These things are context-hungry but most people give them nothing. "Write a function that fixes my problem" doesn't work, surprise surprise.

We need different primitives. Not "read everything the LLM wrote very carefully" ways to feed it the why, the motivation, the discussion and prior art. Otherwise yeah, we're building a mountain of code nobody understands.

You have a similar problem with projects where a large number of offshore developers is used. Every day you get a huge pile of code to review which is basically impossible within the available time. So you end up with a system that nobody really understands.

I find LLMs most useful in this understanding of legacy code.

I can ask questions like, “how is this code organized” and, “where does [thing] happen?”

I've done my share of vibe coding, and I completely agree with OP.

You just don't build up the necessary mental model of what the code does when vibing, and so although you saved time generating the code, you lose all that anyway when you hit a tricky bug and have to spend time building up the mental model to figure out what's wrong.

And saying "oh just do all the planning up front" just doesn't work in the real world where requirements change every minute.

And if you ever see anyone using "accepted lines" as a metric for developer productivity/hours saved, take it with a grain of salt.

Soon a capable LLM will have enough training material to spit out LLMs are atrophying coding skills / LLM code is unmaintainable/ LLM code is closing down opportunities for juniors / LLMs do the fun bits of coding pieces on demand.

A lot of these criticisms are valid and I recognise there's a need for people to put their own personal stake in the ground as being one of the "true craftsmen" but we're now at the point where a lot of these articles are not covering any real new ground.

At least some individual war stories about examples where people have tried to apply LLMs would be nice, as well as not pretending that the problem of sloppy code didn't exist before LLMs.

I started using LLMs to refactor and maintain utility scripts that feed data into one of my database driven websites. I don't see a downside to this sort of use of something like Claude Code or Cursor.

This is not full blown vibe coding of a web application to be sure.

I this the only way to escape this trap is by developing better LLMs in the future. The rapid rate at which new AI-generated code is produced means that humans will no longer be able to review it all.

My experience has been that LLMs help me make sense of new code much faster than before.

When I really need to understand what's happening with code, I generally will write it each step.

LLMs make it much easier for me to do this step and more. I've used LLMs to quickly file PRs for new (to me) code bases.

Well, IMO, the issue is that we are trying to merge with AI/LLM. Why must both of us understand the code base? Before it was us that just understood it, why not just have the AI understand it all? why do you need to understand it? to do what exactly? document it? improve it? fix it? Well, let the LLM do all of that too.

LLMs are powerful tools but they are not going to save the world. I have seen this before. The experienced crowd gets chuffed because it is a new pattern that radically changes their current workflow. The new crowd haven't optimised yet so they over use the new way of doing things until they moderate it. The only difference I can detect is that rate of change increased to an almost uncomprehensable pace.

The wave’s still breaking, so I’m going to ride it out until it smooths into calm water. Maybe it never will. I don't know.

[dead]

A friend was recently telling me about an LLM'd PR he was reviewing submitted by a largely non-technical manager where the feature from the outside entirely appeared to work, but actually investigating the thousands of lines of generated code, it was instead hacking their response cache system to appear to work without actually updating anything on the backend.

It took a ton of effort on his part to convince his manager that this wasn't ready to be merged.

I wonder how much vibe coded software is out there in the wild that just appears to work?