HN Explorer

Does anyone find AI code review useful?

1 points by swimmingpolar ↗ HN
I’m genuinely trying to understand the point of AI code review.

Once an LLM generated change gets into a few hundreds or thousands of lines, a human reviewer can barely read the whole thing in any meaningful way. And even if they can, why should they spend their time reviewing thousands of lines generated by someone else’s AI?

If the code is going to be reviewed by another AI anyway, why doesn’t the developer opening the PR just run one more iteration before submitting it?

Is the assumption that after a thorough implementation pass, having a different AI review the code “some time later” will catch things the original AI somehow missed?

Or is the assumption that my precious "Code Review AI" has some superior prompt and sacred collection of skills that makes it meaningfully different from the AI the developer used to write the code?

If that’s the case, why not just give developers that reviewer AI locally and have them iterate against it before opening the PR?

What’s the point? Why not just blindly merge everything and see if it breaks or not?

10 comments

[ 2.2 ms ] story [ 8.5 ms ] thread
deadcatfound
The value is independence plus policy, not another generic pass. A thousand-line AI PR is already a process failure; keep diffs small and review against explicit invariants.
blinkbat
Cut velocity and increase supervision
spawrks
In my experience there is something beneficial about having a different context review and prompted differently, but it's not used very well in practice. Your previous context does create its own blind spot, but you're also getting at a problem in non AI reviews which is "what am I reviewing for?". Some companies like Google have traditionally had very tight guidelines for what a PR reviewer is supposed to criticize and what not to worry about, I don't think AI code review is much different on this topic
swimmingpolar
With that, we're back to square one again. Why not just provide that set of guidelines, prompts, and skills locally, which is very likely something modern LLMs can already handle?

Can't really see the point of settings up a separate AI code review step and make that into some sort of stage that everybody pretends to care. Just to make it look like something extra is, hopefully, happening?

Plus, I feel like any previous context a developer had before submitting the PR must have the most accurate context rather than a code review AI.

It's more on pure curiosity than ranting.

othmanosx
Merging blindly is bad for you as the owner of the code, LLMs are not that good yet, we’re still finding it make mistales and write slop and our responsibility as engineers is to take ownership and verify it.

The AI reviewers just make this easier for us, I’m not talking about the walls of text it adds as it is exhausting to read (I know) but the fact that it could catch real bugs before you even read the actual code is the benefit, and I’ve integrated this flow into my routine, a loop of a coder and a reviewer taking turns before handing me the results to read myself.

The more I find stuff, the more I improve my own code review skill. And I actually created my initial skill by distilling the code reviews and comments I made myself on github for the past 2 years, training an AI on how I review and give feedback on PRs, that produced a skill that is like another copy of me reviewing the code and refining the code before I read it myself.

ohans
What I've found to work is just to bring the review cycle locally, e.g., implement with Claude and review with Codex (with my guardrails set) in the terminal. Something like Herdr works well; I use Coldtea

That being said, the reason I think cloud AI review companies (that review on PRs) work is that at a large org you need a way to ensure that the review indeed happens. And you can't blindly trust that the dev did this locally.

We're a small team, so shifting left works. Just review locally. If you're a large org, that's a different problem space