Funny article, but it seems that the author did not get the "Definition of Done" memo.
While...
> Writing Code Was Never the Bottleneck
...it was also never the job that needed to get done. We wanted to put well working functionality in the hands of users, in an extendible way (so we could add more features later without too much hassle).
If lines of code were the metric of success (like "deal value" is for sales) we would incentivize developers for lines of code written.
TBH, I feel like the biggest help Cursor gives me is with understanding large-ish legacy codebases. It's an excellent (& active) "rubber duck". So I'm not sure the argument holds - LLMs don't just write code.
I dont think the authors comments are without merit. My experience has shown me issues are usually more upfront and after the fact.
Either the bottleneck between product organizations and engineering on getting decent requirements to know what to build and engineering teams being unwilling to start until they have every I dotted and T crossed.
The backend of the problem is that already most of the code e see written is poorly documented across the spectrum. How many commit messages have we seen of "wip" for instance? Or you go to a repository and the Readme is empty?
So the real danger is the stack overflow effect on steroids. It's not just a block of code that was put in that wasn't understood, its now entire projects, and there's little to no documentation to explain what was done or why decisions were made.
The article misses the point that LLMs are not removing the bottleneck of writing code for people who know how to write code. It's removing this bottleneck for everyone else.
Even without LLMs, we were approaching a point of saturation where software development was bottlenecked by market demand and funding, not by a shortage of code. Our tooling has become so powerful that the pure act of programming is secondary.
It's a world away from when the industry began. There's a great story from Bill Gates about a time when his ability to simply write code was an incredibly scarce resource. A company was so desperate for programmers that they hired him and Paul Allen as teenagers:
"So, they were paying penalties... they said, 'We don’t care [that they are kids].' You know, so I go down there. You know, I’m like 16, but I look about 13. They hire us. They pay us. It’s a really amazing project... they got a kick out of how quickly I could write code."
That story is a powerful reminder of how much has changed. Writing code was the bottleneck years ago. However the core problem has shifted from "How do we build it?" to "What should we build and is there a business for it?"
Right, we all know this. LLMs write a lot of bad code that cannot be realistically reviewed.
I've even had code submitted to me by juniors which didn't make any sense. When I ask them why they did that, they say they don't know, the LLM did it.
What this new trend is doing is generating a lot of noise and overhead on maintenance.
The only way forward, if embracing LLMs, is to use LLMs also for the reviewing and maintenance, which obviously will lead to messy spaghetti, but you now have the tools to manage that.
But the important realization is that for most businesses, quality doesn't really matter. Throwaway LLM code is good enough, and when it isn't you can just add more LLM on top until it does what you think you need.
My most recent example of this is mentoring young, ambitious, but inexperienced interns.
Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before:
- During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "good catch, I'll fix that" (also reminiscent of an LLM).
- The time spent on trivial issues went down a lot, almost zero, the remaining issues were much more subtle and time-consuming to find and describe.
- Many bugs were of a new kind (to me), the code would look like it does the right thing but actually not work at all, or just be much more broken than code with that level of "polish" would normally be. This breakdown of pattern-matching compared to "organic" code made the overhead much higher. Spending decades reviewing code and answering Stack Overflow questions often makes it possible to pinpoint not just a bug but how the author got there in the first place and how to help them avoid similar things in the future.
- A simple, but bad (inefficient, wrong, illegal, ugly, ...) solution is a nice thing to discuss, but the LLM-assisted junior dev often cooks up something much more complex, which can be bad in many ways at once. The culture of slowly growing a PR from a little bit broken, thinking about design and other considerations, until its high quality and ready for a final review doesn't work the same way.
- Instead of fixing the things in the original PR, I'd often get a completely different approach as the response to my first review. Again, often broken in new and subtle ways.
This lead to a kind of effort inversion, where senior devs spent much more time on these PRs than the junior authors themselves. The junior dev would feel (I assume) much more productive and competent, but the response to their work would eventually lack most of the usual enthusiasm or encouragement from senior devs.
How do people work with these issues? One thing that worked well for me initially was to always require a lot of (passing) tests but eventually these tests would suffer from many of the same problems
This shouldn't be surprising to anyone in software development. Regardless of how essential your software is, you can just shit out any stupid-ass thing that vaguely works and you've finished your ticket.
Who thought lazy devs were the bottleneck? The industry needs 8x as much regulation as it has now; they can do whatever they want at the moment lol.
One thing I despise about LLMs is transferring the cognitive load to a machine. It’s just another form of tech debt. And you have repay it pretty fast as the project grows.
my LLM win this year was to give the corporate AI my last year's worth of notes, emails and documents and ask it to write my self review. it did a great job. i'm never writing another one of those stupid bits of psychological torture again
otherwise i'm writing embedded systems. fine, LLM, you hold the scope probe and figure out why that PWM is glitching
The author puts the BLUF: "The actual bottlenecks were, and still are, code reviews, knowledge transfer through mentoring and pairing, testing, debugging, and the human overhead of coordination and communication."
They're not wrong, but they're missing the point. These bottlenecks can be reduced when there are fewer humans involved.
Somewhat cynically:
code reviews: now sometimes there's just one person involved (reviewing LLM code) instead of two (code author + reviewer)
knowledge transfer: fewer people involved means this is less of an overhead
debugging: no change, yet
coordination and communication: fewer people means less overhead
LLMs shift the workload — they don’t remove it: sure, but shifting workload onto automation reduces the people involved
Understanding code is still the hard part: not much change, yet
Teams still rely on trust and shared context: much easier when there are fewer people involved
... and so on.
"Fewer humans involved" remains a high priority goal for a lot of employers. You can never forget that.
> The actual bottlenecks were, and still are, code reviews, knowledge transfer through mentoring and pairing, testing, debugging, and the human overhead of coordination and communication. All of this wrapped inside the labyrinth of tickets, planning meetings, and agile rituals.
Most of these only exist because one person cannot code fast enough to produce all the code. If one programmer was fast enough, you would not need a team and then you wouldn't have coordination and communication overhead and so on.
Has anybody previously had Gantt chart paths "non-code-1 -> code-1 -> non-code-2 =-> code-2" and transformed them into coding tasks, and taking advantage of the newfound coding speed? What did you do? I would need buy-in from people.
Writing software is like a combination of writing a short story, cleaning your room, and planning a vacation. The bottleneck is always low confidence, much like work anywhere else.
I have watched for almost 20 years employers try to solve and cheat their way around this low confidence. The result is always the same: some shitty form of pattern copy/paste, missing originality, and delivery timelines for really basic features. The reasons for this is that nobody wants to invest in training/baselines and great fear that if they do have something perceived as talent that its irreplaceable and can leave.
My current job in enterprise API management is the first time where the bottleneck is different. Clearly the bottleneck is the customer’s low confidence, as opposed to the developers, and manifests as a very slow requirements gathering process.
I think a lot of teams will wrestle with the existing code review process being abused for quite a while. A lot of people are lazy or get into tech because it’s easy money. The combination of LLMs and a solid code review process means you can submit slop and not even be blamed for the results easier than ever.
In a professional setting, I agree 100%, no notes. Where LLMs have helped me the most are actually side projects. There, writing the code is absolutely the bottleneck - I literally can't (or perhaps won't is more truthful) allocate enough time to write code for the little apps I've thought of to solve some small problem.
Yet another article trying to take away from the impact of LLMs. This one is more subtle than most, but still the message is "this problem that was solved, was never actually the problem."
Except... writing code is often a bottleneck. Yeah, code reviews, understanding the domain, etc, is also a bottleneck. But Cursor lets me write apps and tools in 1/20th the time it would take me in an area where I am an expert. It very much has removed my biggest bottleneck.
I agree with most of this. Writing code is one of the easy bits of Software Development. Writing the specifications about what to write is hard.
Once you can specify what to create, and do it well, then actually creating it is quite cheap.
However, as a software developer that often feel I'm pulled into 10 hours of meetings to argue the benefits of one 2-hour thing over the other 2-hour thing, my view is often "Lets do both and see which one comes out best". The view of less technical participants in meetings is always that development is expensive, so we must at all cost avoid developing the wrong thing.
AI can really take hat equation to the extreme. You can make ten different crappy and non-working proof-of-concept things very cheaply. Then throw them out and manually write (or adapt) the final solution just like you always did. But the hard part wasn't writing the code, it was that meeting where it was decided how it should work. But just like discussing a visual design is helped by having sketches, I think "more code" isn't necessarily bad. AI's produce sub par code very quickly. And there are good uses for that: it's a sketch tool for code.
I saw one tech company say they’re going to measure the impact of AI tools by counting merged pull requests per engineer. Seems like I great recipe for AI bullshit churn counting as positive impact.
91 comments
[ 3.2 ms ] story [ 66.7 ms ] threadWhile...
> Writing Code Was Never the Bottleneck
...it was also never the job that needed to get done. We wanted to put well working functionality in the hands of users, in an extendible way (so we could add more features later without too much hassle).
If lines of code were the metric of success (like "deal value" is for sales) we would incentivize developers for lines of code written.
Either the bottleneck between product organizations and engineering on getting decent requirements to know what to build and engineering teams being unwilling to start until they have every I dotted and T crossed.
The backend of the problem is that already most of the code e see written is poorly documented across the spectrum. How many commit messages have we seen of "wip" for instance? Or you go to a repository and the Readme is empty?
So the real danger is the stack overflow effect on steroids. It's not just a block of code that was put in that wasn't understood, its now entire projects, and there's little to no documentation to explain what was done or why decisions were made.
It's a world away from when the industry began. There's a great story from Bill Gates about a time when his ability to simply write code was an incredibly scarce resource. A company was so desperate for programmers that they hired him and Paul Allen as teenagers:
That story is a powerful reminder of how much has changed. Writing code was the bottleneck years ago. However the core problem has shifted from "How do we build it?" to "What should we build and is there a business for it?"Source: https://youtu.be/H1PgccykclM?si=YuIFsUcWc6sHRkAg
I've even had code submitted to me by juniors which didn't make any sense. When I ask them why they did that, they say they don't know, the LLM did it.
What this new trend is doing is generating a lot of noise and overhead on maintenance. The only way forward, if embracing LLMs, is to use LLMs also for the reviewing and maintenance, which obviously will lead to messy spaghetti, but you now have the tools to manage that.
But the important realization is that for most businesses, quality doesn't really matter. Throwaway LLM code is good enough, and when it isn't you can just add more LLM on top until it does what you think you need.
As I get older I spend more of my coding time on walks, at the whiteboard, reading research, and running experiments
Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before:
- During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "good catch, I'll fix that" (also reminiscent of an LLM).
- The time spent on trivial issues went down a lot, almost zero, the remaining issues were much more subtle and time-consuming to find and describe.
- Many bugs were of a new kind (to me), the code would look like it does the right thing but actually not work at all, or just be much more broken than code with that level of "polish" would normally be. This breakdown of pattern-matching compared to "organic" code made the overhead much higher. Spending decades reviewing code and answering Stack Overflow questions often makes it possible to pinpoint not just a bug but how the author got there in the first place and how to help them avoid similar things in the future.
- A simple, but bad (inefficient, wrong, illegal, ugly, ...) solution is a nice thing to discuss, but the LLM-assisted junior dev often cooks up something much more complex, which can be bad in many ways at once. The culture of slowly growing a PR from a little bit broken, thinking about design and other considerations, until its high quality and ready for a final review doesn't work the same way.
- Instead of fixing the things in the original PR, I'd often get a completely different approach as the response to my first review. Again, often broken in new and subtle ways.
This lead to a kind of effort inversion, where senior devs spent much more time on these PRs than the junior authors themselves. The junior dev would feel (I assume) much more productive and competent, but the response to their work would eventually lack most of the usual enthusiasm or encouragement from senior devs.
How do people work with these issues? One thing that worked well for me initially was to always require a lot of (passing) tests but eventually these tests would suffer from many of the same problems
Who thought lazy devs were the bottleneck? The industry needs 8x as much regulation as it has now; they can do whatever they want at the moment lol.
otherwise i'm writing embedded systems. fine, LLM, you hold the scope probe and figure out why that PWM is glitching
They're not wrong, but they're missing the point. These bottlenecks can be reduced when there are fewer humans involved.
Somewhat cynically:
code reviews: now sometimes there's just one person involved (reviewing LLM code) instead of two (code author + reviewer)
knowledge transfer: fewer people involved means this is less of an overhead
debugging: no change, yet
coordination and communication: fewer people means less overhead
LLMs shift the workload — they don’t remove it: sure, but shifting workload onto automation reduces the people involved
Understanding code is still the hard part: not much change, yet
Teams still rely on trust and shared context: much easier when there are fewer people involved
... and so on.
"Fewer humans involved" remains a high priority goal for a lot of employers. You can never forget that.
Most of these only exist because one person cannot code fast enough to produce all the code. If one programmer was fast enough, you would not need a team and then you wouldn't have coordination and communication overhead and so on.
I have watched for almost 20 years employers try to solve and cheat their way around this low confidence. The result is always the same: some shitty form of pattern copy/paste, missing originality, and delivery timelines for really basic features. The reasons for this is that nobody wants to invest in training/baselines and great fear that if they do have something perceived as talent that its irreplaceable and can leave.
My current job in enterprise API management is the first time where the bottleneck is different. Clearly the bottleneck is the customer’s low confidence, as opposed to the developers, and manifests as a very slow requirements gathering process.
Except... writing code is often a bottleneck. Yeah, code reviews, understanding the domain, etc, is also a bottleneck. But Cursor lets me write apps and tools in 1/20th the time it would take me in an area where I am an expert. It very much has removed my biggest bottleneck.
Once you can specify what to create, and do it well, then actually creating it is quite cheap.
However, as a software developer that often feel I'm pulled into 10 hours of meetings to argue the benefits of one 2-hour thing over the other 2-hour thing, my view is often "Lets do both and see which one comes out best". The view of less technical participants in meetings is always that development is expensive, so we must at all cost avoid developing the wrong thing.
AI can really take hat equation to the extreme. You can make ten different crappy and non-working proof-of-concept things very cheaply. Then throw them out and manually write (or adapt) the final solution just like you always did. But the hard part wasn't writing the code, it was that meeting where it was decided how it should work. But just like discussing a visual design is helped by having sketches, I think "more code" isn't necessarily bad. AI's produce sub par code very quickly. And there are good uses for that: it's a sketch tool for code.