This isn't some triviality you can throw aside as unimportant, it is the shape that the code has today, and limits and controls what it will have tomorrow.
It's how you make things intuitive, and it is equally how you ensure people follow a correct flow and don't trap themselves into a security bug.
Code review is part of the job, but one of the least enjoyable parts. Developers like _writing_ and that gives the most job satisfaction. AI tools are helpful, but inherently increases the amount of code we have to review with more scrutiny than my colleagues because of how unpredictable - yet convincing - it can be. Why did we create tools that do the fun part and increase the non-fun part? Where are the "code-review" agents at?
> If you’re a nitpicky code reviewer, I think you will struggle to use AI tooling effectively. [...] Likewise, if you’re a rubber-stamp code reviewer, you’re probably going to put too much trust in the AI tooling.
So in other words, if you are good at code review you are also good enough at writing code that you will be better off writing it yourself for projects you will be responsible for maintaining long term. This is true for almost all of them if you work at a sane place or actually care about your personal projects. Writing code for you is not a chore and you can write it as fluently and quickly as anything else.
Your time "using AI" is much better spent filling in the blanks when you're unfamiliar with a certain tool or need to discover a new one. In short, you just need a few google searches a day... just like it ever was.
I will admit that modern LLMs have made life easier here. AI summaries on search engines have indeed improved to the point where I almost always get my answer and I no longer get hung up meat-parsing poorly written docs or get nerd-sniped pondering irrelevant information.
I have received a few LLM produced PRs from peers from adjacent teams, in good faith but not familiar with the project, and they increasingly infuriate me. They were all garbage, but there’s a great asymmetry: it costs my peers nothing to generate them, it costs me precious time to refute them. And what can I do really? Saying “it’s irreparable garbage because the syntax might be right but it’s conceptually nonsense” but that’s not the most constructive take.
This idea that you can get good results from a bad process as long as you have good quality control seems… dubious, to say the least. “Sure, it’ll produce endless broken nonsense, but as long as someone is checking, it’s fine.” This, generally, doesn’t really work. You see people _try_ it in industry a bit; have a process which produces a high rate of failures, catch them in QA, rework (the US car industry used to be notorious for this). I don’t know of any case where it has really worked out.
Imagine that your boss came to you, the tech lead of a small team, and said “okay, instead of having five competent people, your team will now have 25 complete idiots. We expect that their random flailing will sometimes produce stuff that kinda works, and it will be your job to review it all.” Now, you would, of course, think that your boss had gone crazy. No-one would expect this to produce good results. But somehow, stick ‘AI’ on this scenario, and a lot of people start to think “hey, maybe that could work.”
Yep. All the process in the world won’t teach you to make a system that works.
The pattern I see over and over is a team aimlessly putting a long through tickets in sprints until an engineer who knows how to solve the problem gets it on track personally.
What I took away from the article was that being good at code review makes the person better at guiding the agent to do the job, giving the right context and constraints at the right time… and not that the code reviewer has to fix whatever agent generated… this is also pretty close to my personal experience… LLM models are a bull which can be guided and definitely not a complete idiot…
In a strange kind of analogy, flowing water can cause a lot of damage.. but a dam built to the right specification and turbines can harness that for something very useful… the art is to learn how to build that dam
Imagine a factory making injection molded plastic toys but instead of pumping out perfect parts 99.999% of the time, the machine gives you 50% and you have to pay people to pull out the bad ones from a full speed assembly line and hope no bad ones get through.
Or more broadly, the existence of complex or any life.
Sure, it's not the way I would pick to do most things, but when your buzzword magical thinking so deep all that you have is a hammer, even if it doesn't look like a nail you will force your wage slaves to hammer it anyway until it works.
As to your other cases.. injection molded plastic parts for things like the spinning t bar spray arm in some dishwashers. Crap molds, pass to low wage or temp to razorblade fix by hand and box up. Personally worked such a temp job before, among others so yes that bad output manual qc and fix up abounds still.
And if we are talking high failure rates... see also chip binning and foundry yields in semiconductors.
Just have to look around to see the dubious seeming is more the norm.
No. The failure conditions of "AI agents" are not even close to classical human mistakes (the only one ones where code review has anything more than an infinitesimal chance to catch). There is absolutely no skill transfer and it is a poor excuse anyway since review was never going to catch anything anyway.
Getting AI to produce a bunch of code and then you having to filter through it all is a massive waste of time. The focus should be on getting AI to produce better code in the first place (e.g., using detailed plans), rather than on the volume of code you can produce...
I have only had real advantages with AI for helping me plan changes, and for it helping me to review my code. Getting it to write code for me has been somewhat helpful, but only for simple tedious changes or first drafts. But it is definitely not something I want to leverage by getting AI to produce more and more code that I then have to filter through and review. No thank you. I feel like this is really the wrong focus for implementing AI into your workflows.
I love doing code review for colleagues since I know that it bolsters our shared knowledge, experience and standards. Code review for an external, stubborn, uncooperative AI? No thanks, that sounds like burnout.
AI-generated code can be useful in the early stages of a project, but it raises concerns in mature ones. Recently, a 280kloc+ Postgres parser was merged into Multigres (https://github.com/multigres/multigres/pull/109) with no public code review. In open source, this is worrying. Many people rely on these projects for learning and reference. Without proper review, AI-generated code weakens their value as teaching tools, and more importantly the trust in pulling as dependencies. Code review isn’t just about bugs, it’s how contributors learn, understand design choices, and build shared knowledge. The issue isn’t speed of building software (although corporations may seem to disagree), but how knowledge is passed on.
> In my view, the best code review is structural. It brings in context from parts of the codebase that the diff didn’t mention.
That may be true for AI code.
But it would be pretty terrible for human-written code to bring this up after the code is written, wasting hours/days effort for lack of a little up-front communication on design.
AI makes routine code generation cheap -- only seconds/minutes and cents are being wasted -- but you essentially still need that design session.
I think that I review code much differently than the author. When I'm reviewing code, my assumption is that the person writing it has already verified that it works. I am primarily looking for readability and code smells.
In an ideal world I'd probably be looking more at the actual logic of the code. However, everywhere I've worked it's a full time job just despirately trying to fight ballooning complexity from people who prioritize quick turn around over quality code.
I am good at code review, sure, but I don't like doing it. It's about as strong an engineering technique as coding at a whiteboard. I know I'm at a tiny fraction of my potential without debugging tools and for that reason code review on github is usually a waste of my time. I'll just write code thanks and I'll move the needle on quality by developing. As a reviewer I'll scan for smells but I assume that you too would be most effective if I left you make and clean up your own messes so long as they aren't egregious
Code review isn't the same as design review, nor are these the only type of things (coding and design) that someone may be trying to use AI for.
If you are going to use AI, and catch it's mistakes, then you need to have expertise in whatever it is you are using the AI for. Even if we limit the discussion just to coding, then being a good code reviewer isn't enough - you'd need to have skill at whatever you are asking the AI to do. One of the valuable things AI can do is help you code using languages and frameworks you are not familiar with, which then of course means you are not going to be competent to review the output, other than in most generic fashion.
A bit off topic, but it's weird to me to see the term "coding" make a comeback in this AI/LLM era. I guess it is useful as a way to describe what AI is good at - coding vs more general software developer, but how many companies nowadays hire coders as opposed to software developers (I know it used to be a thing with some big companies like IBM)? Rather than compartmentalized roles, it seems the direction nowadays is more expecting developers to be able to do everything from business analysis and helping develop requirements, to architecture/design and then full-stack development, and subsequent production support.
code review can be almost as much effort as writing the code, especially when the code is not up to the expectations of the reviewer. this is fine, because you want two people (the original author, and the reviewer) on the code.
when reviewing AI code, not only will the effort needed by the reviewer increase, you also lose the second person (the author) looking at the code, because AI can't do that. it can produce code but not reason about or reflect on it like humans can.
What does this mean for juniors? A few companies are now introducing expectations that all engineers will use coding agents including juniors and grads. If they haven't yet learnt what good looks like through experience how are they going to review code produced by AI agents?
As someone that basically does code reviews for a living, last thing I want to do is code review agents. I want to reduce how much review I’m doing, not hand hold some ai agents.
35 comments
[ 3.3 ms ] story [ 55.0 ms ] thread... Function names compose much of the API.
The API is the structure of the codebase.
This isn't some triviality you can throw aside as unimportant, it is the shape that the code has today, and limits and controls what it will have tomorrow.
It's how you make things intuitive, and it is equally how you ensure people follow a correct flow and don't trap themselves into a security bug.
So in other words, if you are good at code review you are also good enough at writing code that you will be better off writing it yourself for projects you will be responsible for maintaining long term. This is true for almost all of them if you work at a sane place or actually care about your personal projects. Writing code for you is not a chore and you can write it as fluently and quickly as anything else.
Your time "using AI" is much better spent filling in the blanks when you're unfamiliar with a certain tool or need to discover a new one. In short, you just need a few google searches a day... just like it ever was.
I will admit that modern LLMs have made life easier here. AI summaries on search engines have indeed improved to the point where I almost always get my answer and I no longer get hung up meat-parsing poorly written docs or get nerd-sniped pondering irrelevant information.
Imagine that your boss came to you, the tech lead of a small team, and said “okay, instead of having five competent people, your team will now have 25 complete idiots. We expect that their random flailing will sometimes produce stuff that kinda works, and it will be your job to review it all.” Now, you would, of course, think that your boss had gone crazy. No-one would expect this to produce good results. But somehow, stick ‘AI’ on this scenario, and a lot of people start to think “hey, maybe that could work.”
The pattern I see over and over is a team aimlessly putting a long through tickets in sprints until an engineer who knows how to solve the problem gets it on track personally.
In a strange kind of analogy, flowing water can cause a lot of damage.. but a dam built to the right specification and turbines can harness that for something very useful… the art is to learn how to build that dam
Or more broadly, the existence of complex or any life.
Sure, it's not the way I would pick to do most things, but when your buzzword magical thinking so deep all that you have is a hammer, even if it doesn't look like a nail you will force your wage slaves to hammer it anyway until it works.
As to your other cases.. injection molded plastic parts for things like the spinning t bar spray arm in some dishwashers. Crap molds, pass to low wage or temp to razorblade fix by hand and box up. Personally worked such a temp job before, among others so yes that bad output manual qc and fix up abounds still.
And if we are talking high failure rates... see also chip binning and foundry yields in semiconductors.
Just have to look around to see the dubious seeming is more the norm.
- Rewrite it yourself?
- Tell AI to generate it again? — will lead to worse code than the first.
- Write the long prompt (like 6 page) even longer and hope it works this time?
I have only had real advantages with AI for helping me plan changes, and for it helping me to review my code. Getting it to write code for me has been somewhat helpful, but only for simple tedious changes or first drafts. But it is definitely not something I want to leverage by getting AI to produce more and more code that I then have to filter through and review. No thank you. I feel like this is really the wrong focus for implementing AI into your workflows.
1. Give it requirements
2. Tell it to ask me clarifying questions
3. When no more questions, ask it to explain the requirements back to me in a formal PRD
4. I criticize it
5. Tell it to come up with 2 alternative high level designs
6. I pick one and criticize it
7. Tell it to come up with 2 alternative detailed TODO lists
8. I pick one and criticize it
9. Tell it to come up with 2 alternative implementations of one of the TODOs
10. I pick one and criticize it
11. Back to 9
I usually “snapshot” outputs along the way and return to them to reduce useless context.
This is what produces the most decent results for me, which aren’t spectacular but at the very least can be a baseline for my own implementation.
It’s very time consuming and 80% of the time I end up wondering if it would’ve been quicker to just do it all by myself right from the start.
Edit: Reference to the time it took to open the PR: https://www.linkedin.com/posts/sougou_the-largest-multigres-...
That may be true for AI code.
But it would be pretty terrible for human-written code to bring this up after the code is written, wasting hours/days effort for lack of a little up-front communication on design.
AI makes routine code generation cheap -- only seconds/minutes and cents are being wasted -- but you essentially still need that design session.
In an ideal world I'd probably be looking more at the actual logic of the code. However, everywhere I've worked it's a full time job just despirately trying to fight ballooning complexity from people who prioritize quick turn around over quality code.
Code review isn't the same as design review, nor are these the only type of things (coding and design) that someone may be trying to use AI for.
If you are going to use AI, and catch it's mistakes, then you need to have expertise in whatever it is you are using the AI for. Even if we limit the discussion just to coding, then being a good code reviewer isn't enough - you'd need to have skill at whatever you are asking the AI to do. One of the valuable things AI can do is help you code using languages and frameworks you are not familiar with, which then of course means you are not going to be competent to review the output, other than in most generic fashion.
A bit off topic, but it's weird to me to see the term "coding" make a comeback in this AI/LLM era. I guess it is useful as a way to describe what AI is good at - coding vs more general software developer, but how many companies nowadays hire coders as opposed to software developers (I know it used to be a thing with some big companies like IBM)? Rather than compartmentalized roles, it seems the direction nowadays is more expecting developers to be able to do everything from business analysis and helping develop requirements, to architecture/design and then full-stack development, and subsequent production support.
But not me, because I will never touch an agentic tool. And believe me, I have a big smile on my face. Life is good! =D
What’s the best way to review AI code?
I wish I had a local, GitHub PR review-like experience where I can leave comments for the agent.
when reviewing AI code, not only will the effort needed by the reviewer increase, you also lose the second person (the author) looking at the code, because AI can't do that. it can produce code but not reason about or reflect on it like humans can.