Not before coding agents nor after coding agents has any PR taken me 5 hours to review. Is the delay here coordination/communication issues, the "Mythical Mammoth" stuff? I could buy that.
If you save 3 hours building something with agentic engineering and that PR sits in review for the same 30 hours or whatever it would have spent sitting in review if you handwrote it, you’re still saving 3 hours building that thing.
So in that extra time, you can now stack more PRs that still have a 30 hour review time and have more overall throughput (good lord, we better get used to doing more code review)
This doesn’t work if you spend 3 minutes prompting and 27 minutes cleaning up code that would have taken 30 minutes to write anyway, as the article details, but that’s a different failure case imo
I find to be true for expensive approvals as well.
If I can approve something without review, it’s instant. If it requires only immediate manager, it takes a day. Second level takes at least ten days. Third level trivially takes at least a quarter (at least two if approaching the end of the fiscal year). And the largest proposals I’ve pushed through at large companies, going up through the CEO, take over a year.
As they say: an hour of planning saves ten hours of doing.
You don't need so much code or maintenance work if you get better requirements upfront. I'd much rather implement things at the last minute knowing what I'm doing than cave in to the usual incompetent middle manager demands of "starting now to show progress". There's your actual problem.
This is one of the reasons I'm so interested in sandboxing. A great way to reduce the need for review is to have ways of running code that limit the blast radius if the code is bad. Running code in a sandbox can mean that the worst that can happen is a bad output as opposed to a memory leak, security hole or worse.
At some moment I realized that reviews are holding things back most of all. I started to jump to review my team's code ASAP. I started to encourage others to go review things ASAP. It works even in relatively large companies, as long as your team has a reasonable size.
I think the problem is the shape of review processes. People higher up in the corporate food chain are needed to give approval on things. These people also have to manage enormous teams with their own complexities. Getting on their schedule is difficult, and giving you a decision isn't their top priority, slowing down time to market for everything.
So we will need to extract the decision making responsibility from people management and let the Decision maker be exclusively focused on reviewing inputs, approving or rejecting. Under an SLA.
My hypothesis is that the future of work in tech will be a series of these input/output queue reviewers. It's going to be really boring I think. Probably like how it's boring being a factory robot monitor.
Valve is one of the only companies that appears to understand this, as well as that individual productivity is almost always limited by communication bandwidth, and communication burden is exponential as nodes in the tree/mesh grow linearly. [or some derated exponent since it doesn't need to be fully connected]
Actually you can. If you shift the reviews far to the left, and call them code design sessions instead, and you raise problems on dailys, and you pair programme through the gnarly bits, then 90% of what people think a review should find goes away. The expectation that you'll discover bugs and architecture and design problems doesn't exist if you've already agreed with the team what you're going to build. The remain 10% of things like var naming, whitespace, and patterns can be checked with a linter instead of a person. If you can get the team to that level you can stop doing code reviews.
You also need to build a team that you can trust to write the code you agreed you'd write, but if your reviews are there to check someone has done their job well enough then you have bigger problems.
That's partly the point of the article, except the article acknowledges that this is organizationally hard:
> You get things like the famous Toyota Production System where they eliminated the QA phase entirely.
> [This] approach to manufacturing didn’t have any magic bullets. Alas, you can’t just follow his ten-step process and immediately get higher quality engineering. The secret is, you have to get your engineers to engineer higher quality into the whole system, from top to bottom, repeatedly. Continuously.
> The basis of [this system] is trust. Trust among individuals that your boss Really Truly Actually wants to know about every defect, and wants you to stop the line when you find one. Trust among managers that executives were serious about quality. Trust among executives that individuals, given a system that can work and has the right incentives, will produce quality work and spot their own defects, and push the stop button when they need to push it.
> I think we’re going to be stuck with these systems pipeline problems for a long time. Review pipelines — layers of QA — don’t work. Instead, they make you slower while hiding root causes. Hiding causes makes them harder to fix.
If we hired two programmers, the goal was to produce twice the LOC per week. Now we are doing far less than our weekly target. Does not meet expectation.
Anybody has idea on how to avoid childish resistance? Anytime something like this pops up people discuss it into oblivion and teams stay in their old habits
"If you can get the team to that level you can stop doing code reviews."
IMHO / IME (over 20y in dev) reviewing PRs still has value as a sanity check and a guard against (slippery slope) hasty changes that might not have received all of the prior checks you mentioned. A bit of well-justified friction w/ ROI, along the lines of "slow is smooth, smooth is fast".
I've seen this mentioned a couple times lately, so I want to say I don't believe pair programming can serve in place of code review.
Code review benefits from someone coming in fresh, making assumptions and challenging those by looking at the code and documentation. With pair programming, you both take the same logical paths to the end result and I've seen this lead to missing things.
> your reviews are there to check someone has done their job well enough then you have bigger problems
Welcome to working with real people. They go off the rails and ignore everything you’ve agreed to during design because they get lazy or feel schedule pressure and cut corners all the time.
Sideline: I feel like AI obeys the spec better than engineers sometimes sigh.
actually you don't need reviews if you have a realistic enough simulation test environment that is fully instrumentable by the AI agent. If you can simulate it almost exactly as in production and it works, there's no need to code review.
to move to the hyperspeed timescale you need reliable models of verification in the digital realm, fully accessible by AI.
Well we can't not review things, because the workflow demands we review things. So we hacked the process and for big changes we begin by asking people who will be impacted (no-code review), then we do a pre-review of a rough implementation and finally do a formal review in a fraction of the time.
Master planning has never worked for my side projects unless I am building the exact replica of what I've done in the past. The most important decisions are made while I'm deep in the code base and I have a better understanding of the tradeoffs.
I think that's why startups have such an edge over big companies. They can just build and iterate while the big company gets caught up in month-long review processes.
>The expectation that you'll discover bugs and architecture and design problems doesn't exist if you've already agreed with the team what you're going to build.
This is like, there's not going to be surprise on the road you'll take if you already set the destination point. Though most of the time, you are just given a vague description of the kind of place you want to reach, not a precise point targeted. And you are not necessarily starting with a map, not even an outdated one. Also geological forces reshape the landscape at least as fast as you are able to move.
Pairing through the gnarly bits is doing the review, just synchronously and earlier. You haven't eliminated reviews — you've moved them to a higher-bandwidth channel. Genuinely better for catching design-level stuff. But it needs overlapping hours and enough shared context to actually work, which falls apart fast across timezones. Async review exists partly because async work does.
>> Now you either get to spend 27 minutes reviewing the code yourself in a back-and-forth loop with the AI (this is actually kinda fun); or you save 27 minutes and submit unverified code to the code reviewer, who will still take 5 hours like before, but who will now be mad that you’re making them read the slop that you were too lazy to read yourself. Little of value was gained.
This seems to check out, and it's the reason why I can't reconcile the claims of the industry about workers replacement with reality. I still wonder when a reckoning will come, though. seems long overdue in the current environment
> I still wonder when a reckoning will come, though. seems long overdue in the current environment
Never. Until 1-10 person teams starts disrupt enterprises (legacy banks, payments systems, consultancies).
“Why” would you ask? Because it’s a house of cards. If engineers get redundant, then we don’t need teams. If we don’t need teams, then we don’t need team leads/PMs/POs and others, if we don’t need middle management, then we don’t need VPs and others. All of those layers will eventually catch up to what’s going on and kill any productivity gains via bureaucracy.
This reads like a scattered mind with a few good gems, a few assumptions that are incorrect but baked into the author’s world view, and loose coherence tying it all together. I see a lot of myself in it.
I’ll cover one of them: layers of management or bureaucracy does not reduce risk. It creates in-action, which gives the appearance of reducing risk, until some startup comes and gobbles up your lunch. Upper management knows it’s all bullshit and the game theoretic play is to say no to things, because you’re not held accountable if you say no, so they say no and milk the money printer until the company stagnates and dies. Then they repeat at another company (usually with a new title and promotion).
Reviewing things is fast and smooth is things are small. If you have all the involved parties stay in the loop, review happens in the real time. Review is only problematic if you split the do and review steps. The same applies to AI coding, you can chose to pair program with it and then it's actually helpful, or you can have it generate 10k lines of code you have no way of reviewing. You just need people understand that switching context is killing productivity. If more things are happening at the same time and your memory is limited, the time spent on load/save makes it slower than just doing one thing at the time and staying in the loop.
In my experience a culture where teammates prioritise review times (both by checking on updates in GH a few times a day, and by splitting changes agressively into smaller patches) is reflected in much faster overall progress time. It's definitely a culture thing, there's nothing technically or organisationally difficult about implementing it, it just requires people working together considering team velocity more important than personal velocity.
In my experience, good mature organisations have clear review processes to ensure quality, improve collaboration and reduce errors and risk. This is regardless of field. It does slow you down - not 10x - but the benefits outweigh the downsides in the long run.
The worst places I’ve worked have a pattern where someone senior drives a major change without any oversight, review or understanding causing multiple ongoing issues. This problem then gets dumped onto more junior colleagues, at which point it becomes harder and more time consuming to fix (“technical debt”). The senior role then boasts about their successful agile delivery to their superiors who don’t have visibility of the issues, much to the eye-rolls of all the people dealing with the constant problems.
I totally agree with his ideas, but somehow he seems just stating the obvious: startups move better than big orgs and you can solve a problem by dividing it in smaller problems - if possible. And that AI experimentation is cheap.
Nice piece, and rings true. I also think startups and smaller organizations will be able to capture better value out of AI because they simply don't have all those approval layers.
Waiting for a few days of design review is a pain that is easy to avoid: all we need is to be ready to spend a few months building a potentially useless system.
I think this makes an assumption early on which is that things are serialized, when usually they are not.
If I complete a bugfix every 30 minutes, and submit them all for review, then I really don't care whether the review completes 5 hours later. By that time I have fixed 10 more bugs!
Sure, getting review feedback 5 hours later will force me to context switch back to 10 bugs ago and try to remember what that was about, and that might mean spending a few more minutes than necessary. But that time was going to be spent _anyway_ on that bug, even if the review had happened instantly.
The key to keeping speed up in slow async communication is just working on N things at the same time.
Although you'll have to mentally replace the word "agent" with "PR" for it to make sense in this context. The math is the same. It all boils down to how much those context switches costs you. If it's a large cost, then you can get a huge productivity boost by increasing review speed.
In the "show calculations" section, the amount of wasted time caused by context switching is the delta between the numbers in the phrase "T_r adjusted from 30.0 to 35 minutes". That number is increases as context switching cost and "average agent time" (AKA "average PR review time") goes up.
Not sure, but there might be a misunderstanding here:
The value of your bug fix is cashed out only when it reaches the customer, not when you have finished implementing it.
There is a cost of delay for value to reach the customer, and we want that delay to be as short as possible.
So it doesn't matter if you fix 10 bugs because your 10th bug is going to reach production 5x10 hours (that's an exageration, but you get the point) after you had fixed it (which is why the article mentions latency and not touch time)
You can tell me "yes but I also participate in the code review effort in parallel). Yes, but then you are not fixing 10 bugs, you are fixing less and reviewing more, and reviews take longer than implementation (especially with LLMs now in the loop).
It's because of the pretty counter-intuitive Little's Law : the more in-progress you have in parallel, the slower it will get for each item to be completed.
90 comments
[ 2.8 ms ] story [ 89.2 ms ] threadSo in that extra time, you can now stack more PRs that still have a 30 hour review time and have more overall throughput (good lord, we better get used to doing more code review)
This doesn’t work if you spend 3 minutes prompting and 27 minutes cleaning up code that would have taken 30 minutes to write anyway, as the article details, but that’s a different failure case imo
If I can approve something without review, it’s instant. If it requires only immediate manager, it takes a day. Second level takes at least ten days. Third level trivially takes at least a quarter (at least two if approaching the end of the fiscal year). And the largest proposals I’ve pushed through at large companies, going up through the CEO, take over a year.
You don't need so much code or maintenance work if you get better requirements upfront. I'd much rather implement things at the last minute knowing what I'm doing than cave in to the usual incompetent middle manager demands of "starting now to show progress". There's your actual problem.
I agree with him anyway: if every dev felt comfortable hitting a stop button to fix a bug then reviewing might not be needed.
The reality is that any individual dev will get dinged for not meeting a release objective.
At some moment I realized that reviews are holding things back most of all. I started to jump to review my team's code ASAP. I started to encourage others to go review things ASAP. It works even in relatively large companies, as long as your team has a reasonable size.
This can be learned, taught, and instilled.
So we will need to extract the decision making responsibility from people management and let the Decision maker be exclusively focused on reviewing inputs, approving or rejecting. Under an SLA.
My hypothesis is that the future of work in tech will be a series of these input/output queue reviewers. It's going to be really boring I think. Probably like how it's boring being a factory robot monitor.
Actually you can. If you shift the reviews far to the left, and call them code design sessions instead, and you raise problems on dailys, and you pair programme through the gnarly bits, then 90% of what people think a review should find goes away. The expectation that you'll discover bugs and architecture and design problems doesn't exist if you've already agreed with the team what you're going to build. The remain 10% of things like var naming, whitespace, and patterns can be checked with a linter instead of a person. If you can get the team to that level you can stop doing code reviews.
You also need to build a team that you can trust to write the code you agreed you'd write, but if your reviews are there to check someone has done their job well enough then you have bigger problems.
> You get things like the famous Toyota Production System where they eliminated the QA phase entirely.
> [This] approach to manufacturing didn’t have any magic bullets. Alas, you can’t just follow his ten-step process and immediately get higher quality engineering. The secret is, you have to get your engineers to engineer higher quality into the whole system, from top to bottom, repeatedly. Continuously.
> The basis of [this system] is trust. Trust among individuals that your boss Really Truly Actually wants to know about every defect, and wants you to stop the line when you find one. Trust among managers that executives were serious about quality. Trust among executives that individuals, given a system that can work and has the right incentives, will produce quality work and spot their own defects, and push the stop button when they need to push it.
> I think we’re going to be stuck with these systems pipeline problems for a long time. Review pipelines — layers of QA — don’t work. Instead, they make you slower while hiding root causes. Hiding causes makes them harder to fix.
If we hired two programmers, the goal was to produce twice the LOC per week. Now we are doing far less than our weekly target. Does not meet expectation.
IMHO / IME (over 20y in dev) reviewing PRs still has value as a sanity check and a guard against (slippery slope) hasty changes that might not have received all of the prior checks you mentioned. A bit of well-justified friction w/ ROI, along the lines of "slow is smooth, smooth is fast".
Code review benefits from someone coming in fresh, making assumptions and challenging those by looking at the code and documentation. With pair programming, you both take the same logical paths to the end result and I've seen this lead to missing things.
Welcome to working with real people. They go off the rails and ignore everything you’ve agreed to during design because they get lazy or feel schedule pressure and cut corners all the time.
Sideline: I feel like AI obeys the spec better than engineers sometimes sigh.
to move to the hyperspeed timescale you need reliable models of verification in the digital realm, fully accessible by AI.
I think that's why startups have such an edge over big companies. They can just build and iterate while the big company gets caught up in month-long review processes.
This is like, there's not going to be surprise on the road you'll take if you already set the destination point. Though most of the time, you are just given a vague description of the kind of place you want to reach, not a precise point targeted. And you are not necessarily starting with a map, not even an outdated one. Also geological forces reshape the landscape at least as fast as you are able to move.
> Get it code reviewed by the peer next to you 300 minutes → 5 hours → half a day
Is it takes 5 hours for a peer to review a simple bugfix your operation is dysfunctional.
This seems to check out, and it's the reason why I can't reconcile the claims of the industry about workers replacement with reality. I still wonder when a reckoning will come, though. seems long overdue in the current environment
Never. Until 1-10 person teams starts disrupt enterprises (legacy banks, payments systems, consultancies).
“Why” would you ask? Because it’s a house of cards. If engineers get redundant, then we don’t need teams. If we don’t need teams, then we don’t need team leads/PMs/POs and others, if we don’t need middle management, then we don’t need VPs and others. All of those layers will eventually catch up to what’s going on and kill any productivity gains via bureaucracy.
I’ll cover one of them: layers of management or bureaucracy does not reduce risk. It creates in-action, which gives the appearance of reducing risk, until some startup comes and gobbles up your lunch. Upper management knows it’s all bullshit and the game theoretic play is to say no to things, because you’re not held accountable if you say no, so they say no and milk the money printer until the company stagnates and dies. Then they repeat at another company (usually with a new title and promotion).
With AI my task to review is to see high level design choices and forget reviewing low level details. It’s much simpler.
The worst places I’ve worked have a pattern where someone senior drives a major change without any oversight, review or understanding causing multiple ongoing issues. This problem then gets dumped onto more junior colleagues, at which point it becomes harder and more time consuming to fix (“technical debt”). The senior role then boasts about their successful agile delivery to their superiors who don’t have visibility of the issues, much to the eye-rolls of all the people dealing with the constant problems.
See this rarely known trick! You can be up to 9x more efficient if you code something else when you wait for review
> AI
projectile vomits
Fuck engineering, let's work on methods to make artificial retard be more efficient!
Context switch alone would kill any productivity gains from this. And I’m not even touching on conflicting MRs and interdependencies yet.
> you can’t overcome latency with brute force
Curious what rang true to you if not the main point?
1. Whoa, I produced this prototype so fast! I have super powers!
2. This prototype is getting buggy. I’ll tell the AI to fix the bugs.
3. Hmm, every change now causes as many new bugs as it fixes.
4. Aha! But if I have an AI agent also review the code, it can find its own bugs!
5. Wait, why am I personally passing data back and forth between agents
6. I need an agent framework
7. I can have my agent write an agent framework!
8. Return to step 1
the author seems to imply this is recursive when it isn't. when you have an effective agent framework you can ship more high quality code quickly.
If I complete a bugfix every 30 minutes, and submit them all for review, then I really don't care whether the review completes 5 hours later. By that time I have fixed 10 more bugs!
Sure, getting review feedback 5 hours later will force me to context switch back to 10 bugs ago and try to remember what that was about, and that might mean spending a few more minutes than necessary. But that time was going to be spent _anyway_ on that bug, even if the review had happened instantly.
The key to keeping speed up in slow async communication is just working on N things at the same time.
Although you'll have to mentally replace the word "agent" with "PR" for it to make sense in this context. The math is the same. It all boils down to how much those context switches costs you. If it's a large cost, then you can get a huge productivity boost by increasing review speed.
In the "show calculations" section, the amount of wasted time caused by context switching is the delta between the numbers in the phrase "T_r adjusted from 30.0 to 35 minutes". That number is increases as context switching cost and "average agent time" (AKA "average PR review time") goes up.
The value of your bug fix is cashed out only when it reaches the customer, not when you have finished implementing it.
There is a cost of delay for value to reach the customer, and we want that delay to be as short as possible.
So it doesn't matter if you fix 10 bugs because your 10th bug is going to reach production 5x10 hours (that's an exageration, but you get the point) after you had fixed it (which is why the article mentions latency and not touch time)
You can tell me "yes but I also participate in the code review effort in parallel). Yes, but then you are not fixing 10 bugs, you are fixing less and reviewing more, and reviews take longer than implementation (especially with LLMs now in the loop).
It's because of the pretty counter-intuitive Little's Law : the more in-progress you have in parallel, the slower it will get for each item to be completed.