The article throws out a lot of potential issues with AI generated code, but doesn't stop for a moment to consider if solutions currently exist or might exist to these problems.
- Before LLMs, provided that your team did some of their due diligence, you could always expect to have some help when tackling new code-bases.
Has the author never worked on legacy code before?
- (oh, and it has forgotten everything about the initial writing process by then).
Does the author not think this can ever be solved?
- Because of the situation of a Bus Factor of zero that it creates, vibe coding is fundamentally flawed. That is, only until there is an AI that can generate 100% accurate code 100% of the time, and it is fed 100% accurate prompts.
Why does it require 100% accuracy 100% of the time? Humans are not 100% accurate 100% of the time and we seem to trust them with our code.
It's potentially the opposite. If you instrument a codebase with documentation and configuration for AI agents to work well in it, then in a year, that agent will be able to do that same work just as well (or better with model progress) at adding new features.
This assumes your adding documentation, tests, instructions, and other scaffolding along the way, of course.
By that same logic, if a project is documented so thoroughly that an agent could handle all the work, then the bus factor effectively becomes infinite.
Unfortunately the corporate machine has been converging on a bus factor of 0. I've been part of multiple teams now where I was the only one holding knowledge over critical subsystems and whenever I attempted to train people on it, it was futile. Mainly because they would get laid off doing 'cost-savings measures'.
There were times where I was close to getting fed up and just quitting during some of the high profile ops I had to deal with which would've left the entire system inoperable for an extended period of time. And frankly from talking to a lot of other engineers, it sounds like a lot of companies operate in this manner.
I fully expect a lot of these issues to come home to roost as AI compounds loss of institutional knowledge and leads to rapid system decay.
If you're using llms to shit out large swathes of unreviewed code you're doing it wrong and your project is indeed doomed to become unmaintainable the minute it goes down a wrong path architecturally, or you get a bug with complex causes or whatever.
Where llms excel is in situations like:
* I have <special snowflake pile of existing data structures> that I want to apply <well known algorithm> to - bam, half a days work done in 2 minutes.
* I want to set up test data and the bones of unit tests for <complicated thing with lots of dependencies> - bam, half a days work done in 2 minutes (note I said to use the llms for a starting point - don't generate your actual test cases with it, at least not without very careful review - I've seen a lot of really dumb ai generated unit tests).
* I want a visual web editor for <special snowflake pile of existing data structures> that saves to an sqlite db and has a separate backend api, bam 3 days work done in 2 minutes.
* I want to apply some repetitive change across a large codebase that's just too complicated for a clever regex, bam work you literally would have never bothered to do before done in 2 minutes.
You don't need to solve hard problems to massively increase your productivity with llms, you just need to shave yaks. Even when it's not a time save, it still lets you focus mental effort on interesting problems rather than burning out on endless chores.
The folks that think they can now suddenly program without any experience, and not need understand how their product works, are suffering from Dunning-Kruger syndrome. Actually, it is a much broader segment and includes product managers, executives, VCs and the general public.
I agree with the premise and the conclusion, but over almost 20 years of writing, adapting and delivering software I've more than once been in exactly the same situation. Noone to ask, the only person even vaguely familiar with software development left half a year ago. Half of the processes have changed since the software was written, and the people who owned them have left, too.
So while I agree that LLMs will accelerate this process, in my opinion it's not a new flavor, just more of an existing problem. Glad to see this kind of thinking though.
There is a really important point here, and it's critical to be aware of it, but we're really just at the beginning of these tools and workflows and these issues can be solved, IMO, possibly better than with humans.
I've been trying to use LLMs to help code more to mixed success, honestly, but it's clear that they're very good at some things, and pretty bad at others. One of the things they good at obviously is producing lots of text, two important others are that they can be very persistent and thorough.
Producing a lot of code can be a liability, but an LLM won't get annoyed at you if you ask it for thorough comments and updates to docs, READMEs, and ADRs. It'll "happily" document what it just did and "why" - to the degree of accuracy that they're able, of course.
So it's conceivable to me at least, that with the right guidance and structure an LLM-generated codebase might be easier to come into cold years later, for both humans and future LLMs, because it could have excellent documentation.
just not even ten years ago the discussion here was all about "software engineering" trying to be more legitimized as a formal engineering practice, if there should be licensing, if there should be certifications, lots of threads about formal methods to prove algorithms work, and look where we are now. Arguing if humans should even care look at the code we are producing and shipping. crazy shit man
The problem is that our brains really don't like expending calories on information we don't repeatedly use so the further you get from something, the less you understand it or remember it.
So even if you aren't even vibe coding and are trying to review every change, your skills are atrophying. We see this all the time as engineers enter management, they become super competent at the new skills the role requires but quickly become pretty useless at solving technical problems.
It's similar to why it's so hard to go from level2 to level5 in driving automation. We're not really designed to be partially involved in a process - we quickly loose attention, become lazy, and blindly trust the machine. Which is maybe fine if the machine is 100% reliable but we know that isn't the case.
The Bus Factor was an issue long before LLM-generated code. Very few companies structure work to allow a pool of >1 individuals to understand/contribute to it. What I found is -- when companies are well structured with multiple smart individuals per area, the output expectation just ends up creeping up until again there is too much to really know. You can only get away from this with really good engineering management that specifically tries to move people around the codebase and trade-off speed in the process. I have tried to do this, but sometimes pressure from the stakeholders for speed is just too great to do it perfectly.
Shameful plug, i've been writing a book on this with my retrospective as a CTO building like this. I just updated it so you can choose your price (even $0) to make this a less shameful plug on HN: https://ctoretrospective.gumroad.com/l/own-your-system
I dont think anyone has the perfect answer, yet, but LLM-built systems arent that different from having the system built by 10 diff people on eLance/Upwork/Fiverr...so the principles are the same.
I’ve got a new project I’ve been handling with Claude code. Up until now I’ve always pair coded with AIs, so I would know (and usually tweak) every bit of code generated. Now with the agent, it’s easy to miss what’s being made.
Ive been trying to resolve this with things like “make a notebook that walks through this modules functions”, etc, to make it easier for me to review.
In the spirit of literate riding though, why but have these agents spend more time (tokens…money) walking you through what they made.
Likewise, if dev A vibe codes something and leaves it to dev B to maintain, we should think about what AI workflows can get B up to speed fast. “Give me a tour of the code”
LLMs are bad for bad programmers. LLMs will make a bad programmer worse and make a layperson think they're a prodigy.
Meanwhile, the truly skilled programmers are using LLMs to great success. You can get a huge amount of value and productivity from an LLM if and only if you have the skill to do it yourself in the first place.
LLMs are not a tool that magically makes anyone a good programmer. Expecting that to be the case is exactly why they don't work for you. You must already be a good programmer to use these tools effectively.
I have no idea what this will do to the rising generation of programmers and engineers. Frankly I'm terrified for them.
> The only thing you can rely on is on your ability to decipher what a highly imperfect system generated, and maybe ask explanations to that same imperfect system about your code its code (oh, and it has forgotten everything about the initial writing process by then).
This just sounds like every other story I hear about working on ossified code bases as it is. At least AI can ingest large amounts of code quickly, even if as of today it can't be trusted to actually decipher it all.
I recently joined a team with a very messy codebase. The devs were long gone, and even the ones maintaining it didn’t really understand large parts of the code. The bus factor was effectively zero.
What surprised me was how useful AI was. It helped me not only understand the code but also infer the probable intent behind it, which made debugging much faster. I started generating documentation directly from the code itself.
For me, this was a big win. Code is the source of truth. Developer documentation and even shared knowledge are often full of bias, selective memory, or the “Chinese whispers” problem where the story shifts every time it’s retold and never documented. Code doesn’t lie, it just needs interpretation. Using AI to cut through the noise and let the code explain itself felt like a net positive.
45 comments
[ 4.4 ms ] story [ 60.9 ms ] thread- Before LLMs, provided that your team did some of their due diligence, you could always expect to have some help when tackling new code-bases.
Has the author never worked on legacy code before?
- (oh, and it has forgotten everything about the initial writing process by then).
Does the author not think this can ever be solved?
- Because of the situation of a Bus Factor of zero that it creates, vibe coding is fundamentally flawed. That is, only until there is an AI that can generate 100% accurate code 100% of the time, and it is fed 100% accurate prompts.
Why does it require 100% accuracy 100% of the time? Humans are not 100% accurate 100% of the time and we seem to trust them with our code.
"Just ask the author of that perl file".
Sure, it was last edited in -97 and the author is the regional manager now, should I just book a meeting or...?
This assumes your adding documentation, tests, instructions, and other scaffolding along the way, of course.
There were times where I was close to getting fed up and just quitting during some of the high profile ops I had to deal with which would've left the entire system inoperable for an extended period of time. And frankly from talking to a lot of other engineers, it sounds like a lot of companies operate in this manner.
I fully expect a lot of these issues to come home to roost as AI compounds loss of institutional knowledge and leads to rapid system decay.
Humans (and I strongly suspect LLMs, since they're statistical synthesis of human production) are fairly predictable.
We tend to tackle the same problems the same way. So how something is solved, tells you a lot about why, who and when it was solved.
Still, it's a valid point that much of the knowledge is now obscured, but that could be said too of a high employee churn organization.
Where llms excel is in situations like:
* I have <special snowflake pile of existing data structures> that I want to apply <well known algorithm> to - bam, half a days work done in 2 minutes.
* I want to set up test data and the bones of unit tests for <complicated thing with lots of dependencies> - bam, half a days work done in 2 minutes (note I said to use the llms for a starting point - don't generate your actual test cases with it, at least not without very careful review - I've seen a lot of really dumb ai generated unit tests).
* I want a visual web editor for <special snowflake pile of existing data structures> that saves to an sqlite db and has a separate backend api, bam 3 days work done in 2 minutes.
* I want to apply some repetitive change across a large codebase that's just too complicated for a clever regex, bam work you literally would have never bothered to do before done in 2 minutes.
You don't need to solve hard problems to massively increase your productivity with llms, you just need to shave yaks. Even when it's not a time save, it still lets you focus mental effort on interesting problems rather than burning out on endless chores.
How many of you have asked about a process and been told that nobody knows how it works because the person who developed it left the company?
There was a blog post at the top of HN about this, like, yesterday.
I hate the current AI hype hothouse and everything it seems to be doing to the industry... but I couldn't help but laugh.
The post is great. Bus factor of zero is a great coinage.
https://news.ycombinator.com/item?id=43888225
I've been trying to use LLMs to help code more to mixed success, honestly, but it's clear that they're very good at some things, and pretty bad at others. One of the things they good at obviously is producing lots of text, two important others are that they can be very persistent and thorough.
Producing a lot of code can be a liability, but an LLM won't get annoyed at you if you ask it for thorough comments and updates to docs, READMEs, and ADRs. It'll "happily" document what it just did and "why" - to the degree of accuracy that they're able, of course.
So it's conceivable to me at least, that with the right guidance and structure an LLM-generated codebase might be easier to come into cold years later, for both humans and future LLMs, because it could have excellent documentation.
So even if you aren't even vibe coding and are trying to review every change, your skills are atrophying. We see this all the time as engineers enter management, they become super competent at the new skills the role requires but quickly become pretty useless at solving technical problems.
It's similar to why it's so hard to go from level2 to level5 in driving automation. We're not really designed to be partially involved in a process - we quickly loose attention, become lazy, and blindly trust the machine. Which is maybe fine if the machine is 100% reliable but we know that isn't the case.
Shameful plug, i've been writing a book on this with my retrospective as a CTO building like this. I just updated it so you can choose your price (even $0) to make this a less shameful plug on HN: https://ctoretrospective.gumroad.com/l/own-your-system
I dont think anyone has the perfect answer, yet, but LLM-built systems arent that different from having the system built by 10 diff people on eLance/Upwork/Fiverr...so the principles are the same.
Ive been trying to resolve this with things like “make a notebook that walks through this modules functions”, etc, to make it easier for me to review.
In the spirit of literate riding though, why but have these agents spend more time (tokens…money) walking you through what they made.
Likewise, if dev A vibe codes something and leaves it to dev B to maintain, we should think about what AI workflows can get B up to speed fast. “Give me a tour of the code”
LLMs are bad for bad programmers. LLMs will make a bad programmer worse and make a layperson think they're a prodigy.
Meanwhile, the truly skilled programmers are using LLMs to great success. You can get a huge amount of value and productivity from an LLM if and only if you have the skill to do it yourself in the first place.
LLMs are not a tool that magically makes anyone a good programmer. Expecting that to be the case is exactly why they don't work for you. You must already be a good programmer to use these tools effectively.
I have no idea what this will do to the rising generation of programmers and engineers. Frankly I'm terrified for them.
This just sounds like every other story I hear about working on ossified code bases as it is. At least AI can ingest large amounts of code quickly, even if as of today it can't be trusted to actually decipher it all.
What surprised me was how useful AI was. It helped me not only understand the code but also infer the probable intent behind it, which made debugging much faster. I started generating documentation directly from the code itself.
For me, this was a big win. Code is the source of truth. Developer documentation and even shared knowledge are often full of bias, selective memory, or the “Chinese whispers” problem where the story shifts every time it’s retold and never documented. Code doesn’t lie, it just needs interpretation. Using AI to cut through the noise and let the code explain itself felt like a net positive.