HN Explorer

Reviewing in the Age of AI

9 points by maxalbarello ↗ HN
How are you guys ensuring that the code that goes into production is high quality now that the time to review code is significantly greater than the time to generate it.

There is a huge asymmetry between who is generating the code and who is reviewing it, making the review process even more painful than it used to be.

Wondering whether if instead of reviewing PRs, we should instead move towards reviewing plans so that no code is generated before at least another person approves the plan.

Once the code is generated, the users who contributed to the plan can still review it but the fact that both participated in the plan should help reduce the asymmetry.

Feels like we need a way to collaborate and iterate on plans. Would love your thoughts on this.

9 comments

[ 3.1 ms ] story [ 40.9 ms ] thread
ilbert
In the last couple of companies I've worked in, I've felt both overwhelmed by PRs to review and disappointed by my teammates that were just rubber-stamping my PRs
taeshdas
In these times i would argue on probably getting the code reviewed by an ai agent which has been specifically trained on code quality, robustness and company or product specific code practices, that reduces the loan on manual reviewing by a large margin
MatrixOrigin
you've identified a real structural probiem with how most teams are still operating.And we're working on it.
maxalbarello
What are you working on? Curious to hear more
bnchrch
Our teams efforts have shifted toward reviewing plans as a team.

Before code hits a humans eyes it's went through a few independant review passes from our review agents.

If it's low complexity / blast radius it gets auto merged.

If it's high complexity / blast radius it gets flagged for human review.

And funny enough our team has agreed even our human review layer has the best results if we have an agent create supplementary descriptions of the code and potential issues that we read and apply human judgement.

Reason being is that it can be very pendantic. Both in a good and bad way. Either flagging things that are non-concerns, or catching things my lazy human eye wouldnt have caught. e.g. A docstring mis-describing the actual shape of an object.

maxalbarello
interesting! how do you review each other's plans? do you just copy-paste them into chat or something else?
BullForgeLabs
yeah the tricky part with AI generated code is knowing what to actually review. most non-technical folks don't catch the stuff that'll bite you later. honestly just focusing on the boring parts helps. does it crash? does it eat battery? does it actually save your data when you close it? run it on an old device if you can. the fancy code stuff matters less than whether it works when someone's internet is spotty.