AI tends to slow us down because we don't really know what it's good at. Can it write a proper Nginx config? I don't know—let's try. And then we end up wasting 30 minutes on it.
Fully autonomous coding tools like v0, a0, or Aider work well as long as the context is small. But once the context grows—usually due to mistakes made in earlier steps—they just can’t keep up. There's no real benefit of "try again" loop yet.
For now, I think simple VSCode extensions are the most useful. You get focused assistance on small files or snippets you’re working on, and that’s usually all you need.
```
It's common for engineers to end up working on projects which they don't have an accurate mental model of. Projects built by people who have long since left the company for pastures new. It's equally common for developers to work in environments where little value is placed on understanding systems, but a lot of value is placed on quickly delivering changes that mostly work. In this context, I think that AI tools have more of an advantage. They can ingest the unfamiliar codebase faster than any human can, and can often generate changes that will essentially work.
```
Reason: you cannot evaluate the work accurately if you have no mental model. If there's a bug given the systems unwritten assumptions you may not catch it.
Having said that it also depends on how important it is to be writing bug free code in the given domain I guess.
I like AI particularly for green field stuff and one off scripts as it let's you go faster here. Basically you build up the mental model as you're coding with the AI.
Not sure about whether this breaks down at a certain codebase size though.
What I thought was fascinating, and should be a warning sign to everyone here:
Before beginning the study, the average developer expected about a 20% productivity boost.
After ending the study, the average developer (potentially: you) believed they actually were 20% more productive.
In reality, they were 0% more productive at best, and 40% less productive at worst.
Think about what it would be like to be that developer; off by 60% about your own output.
If you can't even gauge your own output without being 40% off on average, 60% off at worst; be cautious about strong opinions on anything in life. Especially politically.
Edit 1: Also consider, quite terrifyingly, if said developers were in an online group, together, like... here. The one developer who said she thought it made everyone slower (the truth in this particular case), would be unanimously considered an idiot, downvoted to the full -4, even with the benefit of hindsight.
Edit 2: I suppose this goes to show, that even on Hacker News, where there are relatively high-IQ and self-aware individuals present... 95% of the crowd can still possibly be wildly delusional. Stick to your gut, regardless of the crowd, and regardless of who is in it.
Not surprising. Use of LLM has only been helpful in initial exploration of unknown code bases or languages for me.
Using it beyond that is just more work. First parse the broken response, remove any useless junk, have it reprocess with updated query.
It’s a nice tool to have (just as search engines gave us easy access to multiple sources/forums), but its limitations are well known. Trying to use it 100% as intended is a massive waste of time and resources (energy use…)
> They are experienced open source developers, working on their own projects
I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before
Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that manually it would take me a full day or two, especially without experience in the architecture
That immediately sped up my development even more, as now I had better data to test things locally
Then a couple hours later, I had already pushed my first PR. All code following the proper coding style and practices of the existing project and the framework. That PR, would have taken me at least a couple of days and up to 2 weeks to fully manually write out and test
So sure, AI won’t speed everyone or everything up. But at least in this one case, it gave me a huge boost
As I keep going, I expect things to slow down a bit, as the complexity of the project grows. However, it’s also given me the chance to get an amazing jumpstart
One mediocre paper/study (it should not even be called that with all the bias and sample size issues) and now we have to put up with stories re-hashing and dissecting it. I really hope these don't get upvoted more in the future.
16 devs. And they weren't allowed to pick which tasks they used the AI on. Ridiculous. Also using it on "old and >1 million line" codebases and then extrapolating that to software engineering in general.
Writers like this then theorize why AI isn't helpful, then those "theories" get repeated until it feels less like a theory and more like a fact and it all proliferates into an echo chamber of AI isn't a useful tool. There have been too many anecdotes and my own personal experience to ignore that it isn't useful.
It is a tool and you have to learn it to be successful with it.
Great article and I was having very similar thoughts with regards to this productivity study and the "Programming as Theory Building" paper. I'm starting to be convinced that if you are the original author of a program and still have the program's context in the head, you are the asymptote to which any and all AI systems will approach but never surpass: maybe not in terms of raw coding speed, but in terms of understanding the program, its vision of development, its deficiencies and hacks, its context, its users and what they want, the broader culture the program exists in, etc.
I really like how the author then brought up the point that for most daily work we don't have the theory built, even a small fraction of it, and that this may or may not change the equation.
All these studies that show "AI makes developers x% more/less productive" are predicated on the idea that developer "productivity" can be usefully captured in a single objectively measurable number.
Someone on X said that these agentic AI tools (Claude Code, Amp, Gemini Cli) are to programming like the table saw was to hand-made woodworking.
It can make some things faster and better than a human with a saw, but you have to learn how to use them right (or you will loose some fingers).
I personally find that agentic AI tools make me be more ambitious in my projects, I can tackle some things I didn't tthougth about doing before. And I also delegate work that I don't like to them because they are going to do it better and quicker than me. So my mind is free to think on the real problems like architecture, the technical debt balance of my code...
Problem is that there is the temptation of letting the AI agent do everything and just commit the result without understanding YOUR code (yes, it was generated by an AI but if you sign the commit YOU are responsible for that code).
So as with any tool try to take the time to understand how to better use it and see if it works for you.
Hey HN -- study author here! (See previous thread on the paper here [1].)
I think this blog post is an interesting take on one specific factor that is likely contributing to slowdown. We discuss this in the paper [2] in the section "Implicit repository context (C.1.5)" -- check it out if you want to see some developer quotes about this factor.
> This is why AI coding tools, as they exist today, will generally slow someone down if they know what they are doing, and are working on a project that they understand.
I made this point in the other thread discussing the study, but in general, these results being surprising makes it easy to read the paper, find one factor that resonates, and conclude "ah, this one factor probably just explains slowdown." My guess: there is no one factor -- there's a bunch of factors that contribute to this result -- at least 5 seem likely, and at least 9 we can't rule out (see the full factors table on page 11).
> If there are no takers then I might try experimenting on myself.
This sounds super cool! I'd be very excited to see how you set this up + how it turns out... please do shoot me an email (in the paper) if you do this!
> AI slows down open source developers. Peter Naur can teach us why
Nit: I appreciate how hard it is to write short titles summarizing the paper (the graph title is the best I was able to do after a lot of trying) -- but I might have written this "Early-2025 AI slows down experienced open-source developers. Peter Naur can give us more context about one specific factor." It's admittedly less of a catchy-title, but I think getting the qualifications right are really important!
Thanks again for the sweet write-up! I'll hang around in the comments today as well.
Good article and it makes sense. I wish I had sometime in my career worked on a codebase that was possible to be understood without 10 years of experience. Instead most of my development time was spent tracing execution paths through tangles of abstractions in nested objects in 10M LOC legacy codebases. My buddy who introduced me to the job is still doing it today and now uses AI and this has given him the free time to start working on his own side projects. So there's certain types if jobs where AI will certainly speed up your development.
Only one developer in this study had more than 50h of Cursor experience, including time spent using Cursor during the study. That one developer saw a 25% speed improvement.
Everyone else was an absolute Cursor beginner with barely any Cursor experience. I don't find it surprising that using tools they're unfamiliar with slows software engineers down.
I don't think this study can be used to reach any sort of conclusion on use of AI and development speed.
Those of current generation students who have access to ai might become slow over time. Because when things are not readily available then they have to struggle and work harder in that process, at that time I thing human a lot of secondary things ! Now when everything is easily available especially knowledge without knowing how to struggle with basics. It will eventually make kids dumb. But can be opposite also.
Eventually even I become slow even I keep on using chat gpt or gemini.
My main two attempts at using an “agentic” coding workflow were trying to incorporate an Outlook COM interface into my rust code base and to streamline an existing abstract windows API interaction to avoid copying memory a couple of times. Both wasted tremendous amounts of time and were ultimately abandoned leaving me only slightly more educated about windows development. They make great autocompletion engines but I just cannot see them being useful in my project otherwise.
I've gotten some pretty cool things working with LLMs doing most of the heavy lifting using the following approaches:
* spec out project goals and relevant context in a README and spec out all components; have the AI build out each component and compose them. I understand the high-level but don't necessarily know all of the low-level details. This is particularly helpful when I'm not deeply familiar with some of the underlying technologies/libraries.
* having an AI write tests for code that I've verified is working. As we all know, testing is tedious - so of course I want to automate it. And we written tests (for well written code) can be pretty easy to review.
I'm one of the regular code reviewers for Burn (a deep learning framework in Rust). I recently had to close a PR because the submitter's bug fix was clearly written entirely by an AI agent. The "fix" simply muted an error instead of addressing the root cause. This is exactly what AI tends to do when it can't identify the actual problem. The code was unnecessarily verbose and even included tests for muting the error. Based on the person's profile, I suspect their motivation was just to get a commit on their record. This is becoming a troubling trend with AI tools.
Typically debugging, e.g., a tricky race condition in an unfamiliar code base would require adding logging, refactoring library calls, inspecting existing logs, and even rewriting parts of your program to be more modular or understandable. This is part of the theory-building.
When you have an AI that says "here is the race condition and here is the code change to make to fix it", that might be "faster" in the immediate sense, but it means you aren't understanding the program better or making it easier for anyone else to understand. There is also the question of whether this process is sustainable: does an AI-edited program eventually fall so far outside what is "normal" for a program that the AI becomes unable to model correct responses?
What I noticed: AI development constantly breaks my flow. It makes me more tired, and I work for shorter time periods on coding.
It's a myth that you can code a whole day long. I usually do intervals of 1-3 hours for coding, with some breaks in between. Procrastination can even happen on work related things, like reading other project members code/changes for an hour. It has a benefit to some extent, but during this time I don't get my work done.
Agentic AI works the best for me. Small refactoring tasks on a selected code snippet can be helpful, but isn't a huge time saver. The worst are AI code completions (first version Copilot style), they are much more noise then help.
I said this when the linked paper was shared and got downvotes: it's based on early 2025 data. My point isn't that it should be completely up to date, but that how we need to consider it in that context. This is pre Claude 4, Claude Code. Pre Gemini 2.5 even. These models are such a big step up from what came previously.
Just like we put a (2023) on articles here so they are considered in the right context, so too this paper should be. Blanket "AI tools slow sown development" statements with a "look this rigorous paper says so!" is ignoring a key variable: the rate of effectiveness improvement. If said paper evaluated with the current models, the picture would be different. Also in 3 months time. AI tools aren't a static thing that either works or don't indefinitely.
41 comments
[ 4.2 ms ] story [ 52.2 ms ] threadFully autonomous coding tools like v0, a0, or Aider work well as long as the context is small. But once the context grows—usually due to mistakes made in earlier steps—they just can’t keep up. There's no real benefit of "try again" loop yet.
For now, I think simple VSCode extensions are the most useful. You get focused assistance on small files or snippets you’re working on, and that’s usually all you need.
``` It's common for engineers to end up working on projects which they don't have an accurate mental model of. Projects built by people who have long since left the company for pastures new. It's equally common for developers to work in environments where little value is placed on understanding systems, but a lot of value is placed on quickly delivering changes that mostly work. In this context, I think that AI tools have more of an advantage. They can ingest the unfamiliar codebase faster than any human can, and can often generate changes that will essentially work. ```
Reason: you cannot evaluate the work accurately if you have no mental model. If there's a bug given the systems unwritten assumptions you may not catch it.
Having said that it also depends on how important it is to be writing bug free code in the given domain I guess.
I like AI particularly for green field stuff and one off scripts as it let's you go faster here. Basically you build up the mental model as you're coding with the AI.
Not sure about whether this breaks down at a certain codebase size though.
Before beginning the study, the average developer expected about a 20% productivity boost.
After ending the study, the average developer (potentially: you) believed they actually were 20% more productive.
In reality, they were 0% more productive at best, and 40% less productive at worst.
Think about what it would be like to be that developer; off by 60% about your own output.
If you can't even gauge your own output without being 40% off on average, 60% off at worst; be cautious about strong opinions on anything in life. Especially politically.
Edit 1: Also consider, quite terrifyingly, if said developers were in an online group, together, like... here. The one developer who said she thought it made everyone slower (the truth in this particular case), would be unanimously considered an idiot, downvoted to the full -4, even with the benefit of hindsight.
Edit 2: I suppose this goes to show, that even on Hacker News, where there are relatively high-IQ and self-aware individuals present... 95% of the crowd can still possibly be wildly delusional. Stick to your gut, regardless of the crowd, and regardless of who is in it.
Using it beyond that is just more work. First parse the broken response, remove any useless junk, have it reprocess with updated query.
It’s a nice tool to have (just as search engines gave us easy access to multiple sources/forums), but its limitations are well known. Trying to use it 100% as intended is a massive waste of time and resources (energy use…)
I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before
Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that manually it would take me a full day or two, especially without experience in the architecture
That immediately sped up my development even more, as now I had better data to test things locally
Then a couple hours later, I had already pushed my first PR. All code following the proper coding style and practices of the existing project and the framework. That PR, would have taken me at least a couple of days and up to 2 weeks to fully manually write out and test
So sure, AI won’t speed everyone or everything up. But at least in this one case, it gave me a huge boost
As I keep going, I expect things to slow down a bit, as the complexity of the project grows. However, it’s also given me the chance to get an amazing jumpstart
16 devs. And they weren't allowed to pick which tasks they used the AI on. Ridiculous. Also using it on "old and >1 million line" codebases and then extrapolating that to software engineering in general.
Writers like this then theorize why AI isn't helpful, then those "theories" get repeated until it feels less like a theory and more like a fact and it all proliferates into an echo chamber of AI isn't a useful tool. There have been too many anecdotes and my own personal experience to ignore that it isn't useful.
It is a tool and you have to learn it to be successful with it.
I really like how the author then brought up the point that for most daily work we don't have the theory built, even a small fraction of it, and that this may or may not change the equation.
Just one problem with that...
It can make some things faster and better than a human with a saw, but you have to learn how to use them right (or you will loose some fingers).
I personally find that agentic AI tools make me be more ambitious in my projects, I can tackle some things I didn't tthougth about doing before. And I also delegate work that I don't like to them because they are going to do it better and quicker than me. So my mind is free to think on the real problems like architecture, the technical debt balance of my code...
Problem is that there is the temptation of letting the AI agent do everything and just commit the result without understanding YOUR code (yes, it was generated by an AI but if you sign the commit YOU are responsible for that code).
So as with any tool try to take the time to understand how to better use it and see if it works for you.
I think this blog post is an interesting take on one specific factor that is likely contributing to slowdown. We discuss this in the paper [2] in the section "Implicit repository context (C.1.5)" -- check it out if you want to see some developer quotes about this factor.
> This is why AI coding tools, as they exist today, will generally slow someone down if they know what they are doing, and are working on a project that they understand.
I made this point in the other thread discussing the study, but in general, these results being surprising makes it easy to read the paper, find one factor that resonates, and conclude "ah, this one factor probably just explains slowdown." My guess: there is no one factor -- there's a bunch of factors that contribute to this result -- at least 5 seem likely, and at least 9 we can't rule out (see the full factors table on page 11).
> If there are no takers then I might try experimenting on myself.
This sounds super cool! I'd be very excited to see how you set this up + how it turns out... please do shoot me an email (in the paper) if you do this!
> AI slows down open source developers. Peter Naur can teach us why
Nit: I appreciate how hard it is to write short titles summarizing the paper (the graph title is the best I was able to do after a lot of trying) -- but I might have written this "Early-2025 AI slows down experienced open-source developers. Peter Naur can give us more context about one specific factor." It's admittedly less of a catchy-title, but I think getting the qualifications right are really important!
Thanks again for the sweet write-up! I'll hang around in the comments today as well.
[1] https://news.ycombinator.com/item?id=44522772
[2] https://metr.org/Early_2025_AI_Experienced_OS_Devs_Study.pdf
Ehhhh... not so much. It had serious design flaws in both the protocol and the analysis. This blog post is a fairly approachable explanation of what's wrong with it: https://www.argmin.net/p/are-developers-finally-out-of-a-job
Everyone else was an absolute Cursor beginner with barely any Cursor experience. I don't find it surprising that using tools they're unfamiliar with slows software engineers down.
I don't think this study can be used to reach any sort of conclusion on use of AI and development speed.
* spec out project goals and relevant context in a README and spec out all components; have the AI build out each component and compose them. I understand the high-level but don't necessarily know all of the low-level details. This is particularly helpful when I'm not deeply familiar with some of the underlying technologies/libraries. * having an AI write tests for code that I've verified is working. As we all know, testing is tedious - so of course I want to automate it. And we written tests (for well written code) can be pretty easy to review.
When you have an AI that says "here is the race condition and here is the code change to make to fix it", that might be "faster" in the immediate sense, but it means you aren't understanding the program better or making it easier for anyone else to understand. There is also the question of whether this process is sustainable: does an AI-edited program eventually fall so far outside what is "normal" for a program that the AI becomes unable to model correct responses?
It's a myth that you can code a whole day long. I usually do intervals of 1-3 hours for coding, with some breaks in between. Procrastination can even happen on work related things, like reading other project members code/changes for an hour. It has a benefit to some extent, but during this time I don't get my work done.
Agentic AI works the best for me. Small refactoring tasks on a selected code snippet can be helpful, but isn't a huge time saver. The worst are AI code completions (first version Copilot style), they are much more noise then help.
Just like we put a (2023) on articles here so they are considered in the right context, so too this paper should be. Blanket "AI tools slow sown development" statements with a "look this rigorous paper says so!" is ignoring a key variable: the rate of effectiveness improvement. If said paper evaluated with the current models, the picture would be different. Also in 3 months time. AI tools aren't a static thing that either works or don't indefinitely.