"My team is no different—we are producing code at 10x of typical high-velocity team. That's not hyperbole - we've actually collected and analyzed the metrics."
Rofl
"The Cost-Benefit Rebalance"
In here he basically just talks about setting up mock dependencies and introducing intermittent failures into them. Mock dependencies have been around for decades, nothing new here.
It sounds like this test system you set up is as time consuming as solving the actual problems you're trying to solve, so what time are you saving?
"Driving Fast Requires Tighter Feedback Loop"
Yes if you're code-vomiting with agents and your test infrastructure isn't rock solid things will fall apart fast, that's obvious. But setting up a rock solid test infrastructure for your system involves basically solving most of the hard problems in the first place. So again, what? What value are you gaining here?
"The communication bottleneck"
Amazon was doing this when I worked there 12 years ago. We all sat in the same room.
"The gains are real - our team's 10x throughput increase isn't theoretical, it's measurable."
Show the data and proof. Doubt.
Yeah I don't know. This reads like complete nonsense honestly.
Paraphrasing:
"AI will give us huge gains, and we're already seeing it. But our pipelines and testing will need to be way stronger to withstand the massive increase in velocity!"
Velocity to do what? What are you guys even doing?
I switched back to Rails for my side project a month ago and ai coding when doing not too complex stuff has been great. While the old NextJS code base was in shambles.
Before I was still doing a good chunk of the NextJS coding. I’m probably going to be directly coding less than 10% of the code base from here on out. I’m now spending time trying to automate things as much as possible, make my workflow better, and see what things can be coded without me in the loop. The stuff I’m talking about is basic CRUD and scraping/crawling.
For serious coding, I’d think coding yourself and having ai as your pair programmer is still the way to go.
> Instead, we use an approach where a human and AI agent collaborate to produce the code changes. For our team, every commit has an engineer's name attached to it, and that engineer ultimately needs to review and stand behind the code. We use steering rules to setup constraints for how the AI agent should operate within our codebase,
This sounds a lot like Tesla's Fake Self Driving. It self drives right up to the crash, then the user is blamed.
This is the first time I see "steering rules" mentioned. I do something similar with Claude, curious how it looks for them and how they integrate it with Q/Kiro.
This reads like "Hey, we're not vibe coding, but when we do, we're careful!" with hints of "AI coding changes the costs associated with writing code, designing features, and refactoring" sprinkles in to stand out.
The way to code going forward with AI is Test Driven Development. The code itself no longer matters. You give the AI a set of requirements, ie. tests that need to pass, and then let it code whatever way it needs to in order to fulfill those requirements. That's it. The new reality us programmers need to face is that code itself has an exact value of $0. That's because AI can generate it, and with every new iteration of the AI, the internal code will get better. What matters now are the prompts.
I always thought TDD was garbage, but now with AI it's the only thing that makes sense. The code itself doesn't matter at all, the only thing that matters is the tests that will prove to the AI that their code is good enough. It can be dogshit code but if it passes all the tests, then it's "good enough". Then, just wait a few months and then rerun the code generation with a new version of the AI and the code will be better. The humans don't need to know what the code actually is. If they find a bug, write a new test and force the AI to rewrite the code to include the new test.
I think TDD has really found its future now that AI coding is here to stay. Human code doesn't matter anymore and in fact I would wager that modifying AI generated code is as bad and a burden. We will need to make sure the test cases are accurate and describe what the AI needs to generate, but that's it.
Why did you think TDD was garbage? Formalizing a specification is all that test first is. It's just that most devs I know had big egos and believing writing tests was somehow below them. I prefer the "build a little, test a little" approach, personally, but there's nothing inherently wrong with TDD.
My prediction is that in the future, a lot of desperate companies are going to need living, breathing reverse software engineers to aid them because they have lost the ability to understand their own codebases.
Oh, and why is code worth $0? A lot of code is throwaway, but I still got paid to produce it and much of it makes money for the company or saves them money.
I could not disagree more strongly with everything you’ve said in this comment.
> The way to code going forward with AI is Test Driven Development.
No. TDD already collapses under its own weight as a project grows.
> The code itself no longer matters.
No. Definitely no. That’s absurd. You can’t box in a correct solution with guard rails. Especially since, even if you could get something close to that, you would also lose the ability to understand the tests.
> You give the AI a set of requirements, ie. tests that need to pass, and then let it code whatever way it needs to in order to fulfill those requirements. That's it. The new reality us programmers need to face is that code itself has an exact value of $0.
No. The opposite. When code is cheap, understanding and control become expensive. Code a human can understand will be the most valuable going forward.
> That's because AI can generate it, and with every new iteration of the AI, the internal code will get better.
No. All code is technical debt. AI produces code faster. Therefore AI produces bugs faster.
”Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it” -Brian Kernighan
This is literally where we’re at. AI writes code just beyond its ability to fix.
> What matters now are the prompts.
No. This is such a dead end. It’s a roll of the dice, and so we have examples of people who seem to get it to build something faster. That’s like saying there are people who win the lottery. It’s true, and it also says nothing of your ability to repeat their process. Confirmation bias of the wins. But in building something reliable, we care more about the floor (minimum quality) than the ceiling (the peak it can reach sometimes).
> The new reality us programmers need to face is that code itself has an exact value of $0.
This is not new at all. Code has always been a liability. It having $0 value would be a great improvement IMHO.
The value was always in the product regardless of the amount of code in it and regardless of its quality. Customers don’t buy code. (Except of course when the code is the product, which is very unusual nowadays.)
"We have real mock versions of all our dependencies!"
Congratulations, you invented end-to-end testing.
"We have yellow flags when the build breaks!"
Congratulations! You invented backpressure.
Every team has different needs and path dependencies, so settles on a different interpretation of CI/CD and software eng process. Productizing anything in this space is going to be an uphill battle to yank away teams' hard-earned processes.
Productizing process is hard but it's been done before! When paired with a LOT of spruiking it can really progress the field. It's how we got the first CI/CD tools (eg. https://en.wikipedia.org/wiki/CruiseControl) and testing libraries (eg. pytest)
In 2015, I promoted React as lightweight and bug-free, no plumbing 25 paths and callbacks between events, APIs, DOM for a page using jQuery. No late nights and angry clients like other projects. Everything went neatly for two weeks, till the induced demand of features brought us to the old late nights and angry clients baseline. For many, LLMs are yet another "you are now 10x better" blessing like Moore's law, SQL over binary files, Python over C, NPM/Pip over header files, React over jQuery etc.
"For me, roughly 80% of the code I commit these days is written by the AI agent"
Therefore, it is not commited by you, but by you in the name of AI agent and the holy slop.
What to say, I hope that 100x productivity is worth it and you are making tons of money.
If this stuff becomes mainstream, I suggest open source developers stop doing the grind part, stop writing and maintaining cool libraries and just leave all to the productivity guys, let's see how far they get.
Maybe I've seen too many 1000x hacker news..
The biggest thing that stood out to me was that they suddenly started working nonstop, even on weekends…? If AI is so great, why can’t they get a single day off in two months?
When Karpathy wrote Software 2.0 I was super excited.
I naively believed that we'll start building black boxes based on requirements, sets of inputs and outputs, and sudden changes of heart from stakeholders that often happen on a daily basis for many of us and mandates almost complete reimagination of project architecture will simply need another pass of training with new parameters.
Instead the mainstream is pushing hard reality where we mass produce a ton of code until it starts to work within guard rails.
Does it really work? Is it maintainable?
Get out of here. We're moving at 200mph.
As a security researcher, I am both salivating at the potential that the proliferation of TDD and other AI-centric "development" brings for me, and scared for IT at the same time.
Before we just had code that devs don't know how to build securely.
Now we'll have code that the devs don't even know what it's doing internally.
Someone found a critical RCE in your code? Good luck learning your own codebase starting now!
"Oh, but we'll just ask AI to write it again, and the code will (maybe) be different enough that the exact same vuln won't work anymore!" <- some person who is going to be updating their resume soon.
I'm going to repurpose the term, and start calling AI-coding "de-dev".
> Now we'll have code that the devs don't even know what it's doing internally.
I think that has already been true for some time for large projects continuously updated over a long time, and lots of developers entering and leaving the project throughout the years because nobody who has a choice wants to do that demoralizing job for long (I was one of them in the 1990s, the job was later given to an Indian H1B who could not switch to something better easily, not before putting in a few years of torture to have a better resume, and possibly a greencard).
Most famous post here, but I would like to see what e.g. Microsoft's devs would have to say, or Adobe's:
Such code has long been held together by the extensive test suites rather than intimate knowledge of how it all works.
The task of the individual developer is to close bug tickets and add features, not to produce an optimal solution, or even refactoring. They long ago gave up on that as taking too long.
In my opinion, AI-coding is basically gambling. The odds of getting a usable output are way better than piping from /dev/urandom/, but ultimately it's still a probabilistic output of whether what you want is in fact what you get. Pay for some tokens, pull the slots, and hopefully your RCE goes away.
This article is right, but I think it may underplay the changes that could be coming soon. For instance, as the top comment here about TDD points out, the actual code does not matter anymore. This is an astounding claim! And it has naturally received a lot of objections in the replies.
But I think the objections can mostly be overcome with a minor adjustment: You only need to couple TDD with a functional programming style. Functional programming lets you tightly control the context of each coding task, which makes AI models ridiculously good at generating the right code.
Given that, if most of your code is tightly-scoped, well-tested components implementing orthogonal functionality, the actual code within those components will not matter. Only glue code becomes important and that too could become much more amenable to extensive integration testing.
At that point, even the test code may not matter much, just the test-cases. So as a developer you would only really need to review and tweak the test cases. I call this "Test-Case-Only Development" (TCOD?)
The actual code can be completely abstracted away, and your main task becomes design and architecture.
All the downsides that have been mentioned will be true, but also may not matter anymore. E.g. in a large team and large codebase, this will lead to a lot of duplicate code with low cohesion. However, if that code does what it is supposed to and is well-tested, does the duplication matter? DRY was an important principle when the cost of code was high, and so you wanted to have as much leverage as possible via reuse. You also wanted to minimize code because it is a liability (bugs, tech debt, etc.) and testing, which required even more code that still didn't guarantee lack of bugs, was also very expensive.
But now that the cost of code is plummeting, that calculus is shifting too. You can churn out code and tests (including even performance tests, which are always an afterthought, if thought of at all) at unimaginable rates.
And all this while reducing the dependencies of developers on libraries and frameworks and each other. Fewer dependencies means higher velocity. The overall code "goodput" will likely vastly outweight inefficiences like duplication.
Unfortunately, as TFA indicates, there is a huge impedance mismatch with this and the architectures (e.g. most code is OO, not functional), frameworks, and processes we have today. Companies will have to make tough decisions about where they are and where they want to get.
I suspect AI-assisted coding taken to its logical conclusion is going to look very different from what we're used to.
50 comments
[ 5.9 ms ] story [ 74.9 ms ] threadnow AWS guy doing it !
"My team is no different—we are producing code at 10x of typical high-velocity team. That's not hyperbole - we've actually collected and analyzed the metrics."
Rofl
"The Cost-Benefit Rebalance"
In here he basically just talks about setting up mock dependencies and introducing intermittent failures into them. Mock dependencies have been around for decades, nothing new here.
It sounds like this test system you set up is as time consuming as solving the actual problems you're trying to solve, so what time are you saving?
"Driving Fast Requires Tighter Feedback Loop"
Yes if you're code-vomiting with agents and your test infrastructure isn't rock solid things will fall apart fast, that's obvious. But setting up a rock solid test infrastructure for your system involves basically solving most of the hard problems in the first place. So again, what? What value are you gaining here?
"The communication bottleneck"
Amazon was doing this when I worked there 12 years ago. We all sat in the same room.
"The gains are real - our team's 10x throughput increase isn't theoretical, it's measurable."
Show the data and proof. Doubt.
Yeah I don't know. This reads like complete nonsense honestly.
Paraphrasing: "AI will give us huge gains, and we're already seeing it. But our pipelines and testing will need to be way stronger to withstand the massive increase in velocity!"
Velocity to do what? What are you guys even doing?
Amazon is firing 30,000 people by the way.
I switched back to Rails for my side project a month ago and ai coding when doing not too complex stuff has been great. While the old NextJS code base was in shambles.
Before I was still doing a good chunk of the NextJS coding. I’m probably going to be directly coding less than 10% of the code base from here on out. I’m now spending time trying to automate things as much as possible, make my workflow better, and see what things can be coded without me in the loop. The stuff I’m talking about is basic CRUD and scraping/crawling.
For serious coding, I’d think coding yourself and having ai as your pair programmer is still the way to go.
1) Abstract data showing an increase in "productivity" ... CHECK
2) Completely lacking in any information on what was built with that "productivity" ... CHECK
Hilarious to read this on the backend of the most widely publicized AWS failure.
This sounds a lot like Tesla's Fake Self Driving. It self drives right up to the crash, then the user is blamed.
These guys actually seem rattled now.
The way to code going forward with AI is Test Driven Development. The code itself no longer matters. You give the AI a set of requirements, ie. tests that need to pass, and then let it code whatever way it needs to in order to fulfill those requirements. That's it. The new reality us programmers need to face is that code itself has an exact value of $0. That's because AI can generate it, and with every new iteration of the AI, the internal code will get better. What matters now are the prompts.
I always thought TDD was garbage, but now with AI it's the only thing that makes sense. The code itself doesn't matter at all, the only thing that matters is the tests that will prove to the AI that their code is good enough. It can be dogshit code but if it passes all the tests, then it's "good enough". Then, just wait a few months and then rerun the code generation with a new version of the AI and the code will be better. The humans don't need to know what the code actually is. If they find a bug, write a new test and force the AI to rewrite the code to include the new test.
I think TDD has really found its future now that AI coding is here to stay. Human code doesn't matter anymore and in fact I would wager that modifying AI generated code is as bad and a burden. We will need to make sure the test cases are accurate and describe what the AI needs to generate, but that's it.
My prediction is that in the future, a lot of desperate companies are going to need living, breathing reverse software engineers to aid them because they have lost the ability to understand their own codebases.
Oh, and why is code worth $0? A lot of code is throwaway, but I still got paid to produce it and much of it makes money for the company or saves them money.
> The way to code going forward with AI is Test Driven Development.
No. TDD already collapses under its own weight as a project grows.
> The code itself no longer matters.
No. Definitely no. That’s absurd. You can’t box in a correct solution with guard rails. Especially since, even if you could get something close to that, you would also lose the ability to understand the tests.
> You give the AI a set of requirements, ie. tests that need to pass, and then let it code whatever way it needs to in order to fulfill those requirements. That's it. The new reality us programmers need to face is that code itself has an exact value of $0.
No. The opposite. When code is cheap, understanding and control become expensive. Code a human can understand will be the most valuable going forward.
> That's because AI can generate it, and with every new iteration of the AI, the internal code will get better.
No. All code is technical debt. AI produces code faster. Therefore AI produces bugs faster.
”Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it” -Brian Kernighan
This is literally where we’re at. AI writes code just beyond its ability to fix.
> What matters now are the prompts.
No. This is such a dead end. It’s a roll of the dice, and so we have examples of people who seem to get it to build something faster. That’s like saying there are people who win the lottery. It’s true, and it also says nothing of your ability to repeat their process. Confirmation bias of the wins. But in building something reliable, we care more about the floor (minimum quality) than the ceiling (the peak it can reach sometimes).
This is not new at all. Code has always been a liability. It having $0 value would be a great improvement IMHO.
The value was always in the product regardless of the amount of code in it and regardless of its quality. Customers don’t buy code. (Except of course when the code is the product, which is very unusual nowadays.)
Congratulations, you invented end-to-end testing.
"We have yellow flags when the build breaks!"
Congratulations! You invented backpressure.
Every team has different needs and path dependencies, so settles on a different interpretation of CI/CD and software eng process. Productizing anything in this space is going to be an uphill battle to yank away teams' hard-earned processes.
Productizing process is hard but it's been done before! When paired with a LOT of spruiking it can really progress the field. It's how we got the first CI/CD tools (eg. https://en.wikipedia.org/wiki/CruiseControl) and testing libraries (eg. pytest)
So I wish you luck!
Waiting to see anyone show even a month ahead of schedule after 6 months.
I naively believed that we'll start building black boxes based on requirements, sets of inputs and outputs, and sudden changes of heart from stakeholders that often happen on a daily basis for many of us and mandates almost complete reimagination of project architecture will simply need another pass of training with new parameters.
Instead the mainstream is pushing hard reality where we mass produce a ton of code until it starts to work within guard rails.
Before we just had code that devs don't know how to build securely.
Now we'll have code that the devs don't even know what it's doing internally.
Someone found a critical RCE in your code? Good luck learning your own codebase starting now!
"Oh, but we'll just ask AI to write it again, and the code will (maybe) be different enough that the exact same vuln won't work anymore!" <- some person who is going to be updating their resume soon.
I'm going to repurpose the term, and start calling AI-coding "de-dev".
I think that has already been true for some time for large projects continuously updated over a long time, and lots of developers entering and leaving the project throughout the years because nobody who has a choice wants to do that demoralizing job for long (I was one of them in the 1990s, the job was later given to an Indian H1B who could not switch to something better easily, not before putting in a few years of torture to have a better resume, and possibly a greencard).
Most famous post here, but I would like to see what e.g. Microsoft's devs would have to say, or Adobe's:
https://news.ycombinator.com/item?id=18442941
Such code has long been held together by the extensive test suites rather than intimate knowledge of how it all works.
The task of the individual developer is to close bug tickets and add features, not to produce an optimal solution, or even refactoring. They long ago gave up on that as taking too long.
I am working on a legacy project. This is already the case!
Haha, that already happens in almost any project after 2-3 years.
I think he is right.
But I think the objections can mostly be overcome with a minor adjustment: You only need to couple TDD with a functional programming style. Functional programming lets you tightly control the context of each coding task, which makes AI models ridiculously good at generating the right code.
Given that, if most of your code is tightly-scoped, well-tested components implementing orthogonal functionality, the actual code within those components will not matter. Only glue code becomes important and that too could become much more amenable to extensive integration testing.
At that point, even the test code may not matter much, just the test-cases. So as a developer you would only really need to review and tweak the test cases. I call this "Test-Case-Only Development" (TCOD?)
The actual code can be completely abstracted away, and your main task becomes design and architecture.
It's not obvious this could work, largely because it violates every professional instinct we have. But apparently somebody has even already tried it with some success: https://www.linkedin.com/feed/update/urn:li:activity:7196786...
All the downsides that have been mentioned will be true, but also may not matter anymore. E.g. in a large team and large codebase, this will lead to a lot of duplicate code with low cohesion. However, if that code does what it is supposed to and is well-tested, does the duplication matter? DRY was an important principle when the cost of code was high, and so you wanted to have as much leverage as possible via reuse. You also wanted to minimize code because it is a liability (bugs, tech debt, etc.) and testing, which required even more code that still didn't guarantee lack of bugs, was also very expensive.
But now that the cost of code is plummeting, that calculus is shifting too. You can churn out code and tests (including even performance tests, which are always an afterthought, if thought of at all) at unimaginable rates.
And all this while reducing the dependencies of developers on libraries and frameworks and each other. Fewer dependencies means higher velocity. The overall code "goodput" will likely vastly outweight inefficiences like duplication.
Unfortunately, as TFA indicates, there is a huge impedance mismatch with this and the architectures (e.g. most code is OO, not functional), frameworks, and processes we have today. Companies will have to make tough decisions about where they are and where they want to get.
I suspect AI-assisted coding taken to its logical conclusion is going to look very different from what we're used to.
> I suspect AI-assisted coding taken to its logical conclusion is going to look very different from what we're used to.
100%. I now design new libraries so that AI can easily write code for them.
The corollary being. If you can't (through skill or effort) verify don't trust.
If you break this pattern you deserve all the follies that become you as a "professional".