HN Explorer

Ask HN: How to deal with long vibe-coded PRs?

186 points by philippta ↗ HN
Today I came across a PR for a (in theory) relatively simple service.

It span across 9000 LOC, 63 new files, including a DSL parser and much more.

How would you go about reviewing a PR like this?

152 comments

[ 3.5 ms ] story [ 116 ms ] thread
zigcBenx
In my opinion no PR should have so much changes. It's impossible to review such things.

The only exception is some large migration or version upgrade that required lots of files to change.

As far it goes for Vibe coded gigantic PRs It's a straight reject from me.

wengo314
reject outright. ask to split it into reasonable chain of changesets.
JohnFen
I'd just reject it for being ridiculous. It didn't pass the first step of the review process: the sniff test.
yodsanklai
You review it like it wasn't AI generated. That is: ask author to split it in reviewable blocks. Or if you don't have an obligation to review it, you leave it there.
gpm
Eh, ask the author to split it in reviewable blocks if you think there's a chance you actually want a version of the code. More likely if it's introducing tons of complexity to a conceptually simple service you just outright reject it on that basis.

Possibly you reject it with "this seems more suitable for a fork than a contribution to the existing project". After all there's probably at least some reason they want all that complexity and you don't.

resonious
This is it. The fact that the PR was vibe coded isn't the problem, and doesn't need to influence the way you handle it.
lm28469
It 100% is.

Why would I bother reviewing code you didn't write and most likely didn't read ?

f311a
It's the problem. I often have to guide LLMs 2-4 times to properly write 150-300 LOC changes because I see how the code can be simplified or improved.

There is no way that 9000 lines of code are decent. It's also very hard to review them and find bad spots. Why spent your time in the first place? It probably took one hour for a person to generate it, but it will take ten to review and point out hundreds (probably) problems.

Without AI, no one would submit 9000 lines, because that's tens of hours of work which you usually split into logical parts.

ericmcer
It is a huge problem. PR reviews are a big deal, not just for code reasons, but they are one of the best teaching tools for new hires. Good ones take time and mental energy.

Asking me to review a shitty PR that you don't understand is just disrespectful. Not only is it a huge waste of everyones time, you're forcing me to do your work for you (understanding and validating the AI solution) and you aren't learning anything because it isn't your work.

danenania
I’m curious how people would suggest dealing with large self-contained features that can’t be merged to main until they are production-ready, and therefore might become quite large prior to a PR.

While it would be nice to ship this kind of thing in smaller iterative units, that doesn’t always make sense from a product perspective. Sometimes version 0 has bunch of requirements that are non-negotiable and simply need a lot of code to implement. Do you just ask for periodic reviews of the branch along the way?

ericmcer
I will schedule review time with coworkers I trust to go over it with them.

It is about ownership to me. I own my PRs. If I throw garbage out and expect you to fix it I am making you own my PRs. No one wants to be forced to own other peoples work.

userbinator
If you try to inspect and question such code, you will usually quickly run into that realisation that the "author" has basically no idea what the code even does.

"review it like it wasn't AI generated" only applies if you can't tell, which wouldn't be relevant to the original question that assumes it was instantly recognisable as AI slop.

If you use AI and I can't tell you did, then you're using it effectively.

charlieyu1
You are optimistic like the author even cared about the code. Most of the time you get another LLM response on why the code “works”
ashdksnndck
If you ask them to break it into blocks, are they not going to submit 10 more AI-generated PRs (each having its own paragraphs of description and comment spam), which you then have to wade through. Why sink even more time into it?
data-ottawa
I think breaking a big PR up like this is usually fair

Sometimes I get really into a problem and just build. It results in very large PRs.

Marking the PR as a draft epic then breaking it down into a sequence smaller PRs makes it much easier to review. But you can solicit big picture critique there.

I’m also a huge fan of documentation, so each PR needs to be clear, describe the bigger picture, and link back to your epic.

ivanjermakov
My record is 45 comments on a single review. Merge conditions were configured so that every comment must be resolved.

If PR author can satisfy it - I'm fine with it.

latexr
> Or if you don't have an obligation to review it, you leave it there.

Don’t just leave it there, that reflects badly on you and your project and pushes away good contributors. If the PR is inadequate, close it.

devrundown
9000 LOC is way too long for a pull request unless there is some very special circumstance.

I would ask them to break it up into smaller chunks.

rvrs
Enforce stacked PRs, reject PRs over 500-1k LoC (I'd argue even lower, but it's a hard sell)
vasan
Just reflect upon it, see if you gave him less time to complete it. I would just have a meet with him and confront it.
CharlieDigital
Ask the submitter to review and leave their comments first or do a peer code review with them and force them to read the code. It's probably the first time they'll have read the code as well...
groguzt
I really like this, the fact that vibe coded PRs are often bad is that people don't review it themselves first, they just look at the form, and if it looks vaguely similar to what they had in their mind, they'll just hit save and not ask the LLM for corrections
TriangleEdge
Amazon eng did some research and found the number of comments in a code review is proportional to the number of lines changed. Huge CRs get little comments. Small CRs get a lot of comments. At Amazon, it's common to have a 150 to 300 line limit to changes. It depends on the team.

In your case, I'd just reject it and ensure repo merges require your approval.

ekjhgkejhgk
Could you please provide a reference? I couldn't find it.
ChrisMarshallNY
I write full app suites that have less than 9000 LoC. I tend toward fewer, large-ish source files, separated by functional domains.

I once had someone submit a patch (back in the SVN days), that was massive, and touched everything in my system. I applied it, and hundreds of bugs popped up.

I politely declined it, but the submitter got butthurt, anyway. He put a lot of work into it.

999900000999
Reject it and tell them to actually code it.
le-mark
How long was this person working on it? Six months? Anything this big should’ve had some sort of design review. The worst is some junior going off and coding some garbage no one sees for a month.
jonchurch_
We are seeing a lot more drive by PRs in well known open source projects lately. Here is how I responded to a 1k line PR most recently before closing and locking. For context, it was (IMO) a well intentioned PR. It purported to implement a grab bag of perf improvements, caching of various code paths, and a clustering feature

Edit: left out that the user got flamed by non contributors for their apparently AI generated PR and description (rude), in defense of which they did say they were using several AI tools to drive the work. :

We have a performance working group which is the venue for discussing perf based work. Some of your ideas have come up in that venue, please go make issues there to discuss your ideas

my 2 cents on AI output: these tools are very useful, please wield them in such a way that it respects the time of the human who will be reading your output. This is the longest PR description I have ever read and it does not sound like a human wrote it, nor does it sound like a PR description. The PR also does multiple unrelated things in a single 1k line changeset, which is a nonstarter without prior discussion.

I don't doubt your intention is pure, ty for wanting to contribute.

There are norms in open source which are hard to learn from the outside, idk how to fix that, but your efforts here deviate far enough from them in what I assume is naivety that it looks like spam.

siwatanejo
Forget about code for a second. This all depends a lot of what goal does the PR achieve? Does it align with the goals of the project?
foxfired
It's funny just today I published an article with the solution to this problem.

If they don't bother writing the code, why should you bother reading it? Use an LLM to review it, and eventually approve it. Then of course, wait for the customer to complain, and feed the complaint back to the LLM. /s

Large LLM generated PRs are not a solution. They just shift the problem to the next person in the chain.

throwaway106382
You don't.

Was your project asking for all this? No? Reject.

alexdowad
Be tactful and kind, but straightforward about what you can't/don't want to spend time reviewing.

"Thanks for the effort, but my time and energy is limited and I can't practically review this much code, so I'm closing this PR. We are interested in performance improvements, so you are welcome to pick out your #1 best idea for performance improvement, discuss it with the maintainers via ..., and then (possibly) open a focused PR which implements that improvement only."

exclipy
I made a /split-commit prompt that automatically splits a megacommit into smaller commits. I've found this massively helpful for making more reviewable commits. You can either run this yourself or send this to your coworker to have them run it before asking you to re-review it.

Sometimes it doesn't split it among optimal boundaries, but it's usually good enough to help. There's probably room for improvement and extension (eg. re-splitting a branch containing many not-logical commits, moving changes between commits, merging commits, ...) – contributions welcome!

You can install it as a Claude Code plugin here: https://github.com/KevinWuWon/kww-claude-plugins (or just copy out the prompt from the repo into your agent of choice)

jeremyjh
I'd just close it without comment. Or maybe if I'm feeling really generous I'll make a FAQ.md that gives a list of reasons why we'll close PRs without review or comment and link that in the close comments. I don't owe anyone any time on my open source projects. That said, I haven't had this issue yet.