Maintainers and repo owners will get where they want to go the fastest by not referring to what/who "generated" code in a PR.
Discussions about AI/LLM code being a problem solely because AI/LLM is not generally a productive conversation.
Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc.
Additionally, if there isn't a code of conduct, AI policy, or, perhaps most importantly, a policy on how to submit PRs and which are acceptable, it's a huge weakness in a project.
In this case, clearly some feathers were ruffled but cool heads prevailed. Well done in the end..
This reminds me of the "good developers must be good at thinking at multiple levels of abstraction at the same time" quote. The things you notice about these AI kids is they didn't even do the bare minimum to reason about their PR from multiple angles. __Of course__ someone is going to ask why the copyright is there. Better have a good answer, or - locked, come back when you do. Really that simple.
Can we please go back to "You have to make an account on our server to contribute or pull from the git?"
One of the biggest problems is the fact that the public nature of Github means that fixes are worth "Faux Internet Points" and a bunch of doofuses at companies like Google made "social contribution" part of the dumbass employee evaluation process.
Forcing a person to sign up would at least stop people who need "Faux Internet Points" from doing a drive-by.
I am strongly considering abandoning Github for tarball + email to send git patches to.
No centralisation of my code in siloes like Github, I won't have to care about bots making hundreds of requests on my self-hosted Gitea instance, would prove to be a noticeable source of friction to vibe coders, and I don't care about receiving tons of external contributions from whomever.
For serious people, it'll only be a matter of running `git format-patch` and sending me an attachment via email.
There are LLMs with more self-awareness than this guy.
Repeatedly using AI to answer questions about the legitimacy of commits from an AI, to people who are clearly skeptical is breathtakingly dense. At least they're open about it.
I did love the ~"I'll help maintain this trash mountain, but I'll need paying". Classy.
> Damn, I can’t debug OCaml on my Mac because there’s no DWARF info…But, hey, there’s AI and it seems to one-shot fairly complex stuff in different languages, from just a Github issue…My needs are finally taken care of!
So I do believe using an LLM to generate a big feature like OP did can be very useful, so much that I’m expecting to see such cases more frequently soon. Perhaps in the future, everyone will be constantly generating big program/library extensions that are buggy except for their particular usecase, could be swapped with someone else’s non-public extensions that they generated for the same usecase, and must be re-generated each time the main program/library updates. And that’s OK, as long as the code generation doesn’t use too much energy or cause unforeseen problems. Even badly-written code is still useful when it works.
What’s probably not useful is submitting such code as a PR. Even if it works for its original use-case, it almost certainly still has bugs, and even ignoring bugs it adds tech debt (with bugs, the tech debt is significantly worse). Our code already depends on enough libraries that are complicated, buggy, and badly-written, to the extent that they slow development and make some feasible-sounding features infeasible; let’s not make it worse.
I've closed my share of AI-generated PRs on some OSS repositories I maintain. These contributors seem to jump from one project to another, until their contribution is accepted (recognized ?).
I wonder how long the open-source ecosystem will be able to resist this wave. The burden of reviewing AI-generated PRs is already not sustainable for maintainers, and the number of real open-source contributors is decreasing.
Side note: discovering the discussions in this PR is exactly why I love HN. It's like witnessing the changes in our trade in real time.
> I wonder how long the open-source ecosystem will be able to resist this wave. The burden of reviewing AI-generated PRs is already not sustainable for maintainers, and the number of real open-source contributors is decreasing.
I think the burden is on AI fanbois to ship amazing tools in novel projects before they ask projects with reputations to risk it all on their hype.
To deliver a kernel of truth wrapped in a big bale of sarcasm: you're thinking of it all wrong! The maintainers are supposed to also use AI tools to review the PRs. That's much more sustainable and would allow them to merge 13,000 line PRs several times a day, instead of taking weeks/months to discuss every little feature.
The difference here of course is in how impressed you are by AI tools. The OCaml maintainers are not (and rightly so, IMO), whereas the PR submitter thinks they're so totally awesome and leaving tons of productivity on the table because they're scared of progress or insecure about their jobs or whatever.
Maybe OCaml could advance rapidly if they just YOLO merged big ambitious AI generated PRs (after doing AI code reviews) but that would be a high risk move. They have a reputation for being mature, high quality, and (insanely) reasonable. They would torch it very quickly if people knew this was happening and I think most people here would say the results would be predictably bad.
But lets take the submitter's argument at face value. If AI is so awesome, then we should be able to ship code in new projects unhampered by gatekeepers who insist on keeping slow humans in the loop. Or, to paraphrase other AI skeptics, where's all of the shovelware? How come all of these AI fanbois can only think about laundering their contributions through mature projects instead of cranking out amazing new stuff?
Where's my OCaml compiler 100% re-written in Rust that only depends on the Linux kernel ABI? Should cost a few hundred bucks in Claude credits at most?
To be clear, the submitter has gotten the point and said he was taking his scraps and going to make his own sausage (some Lisp thing). The outcome of that project should be very informative.
In this case the PR author (either LLM or person) is "honest" enough to leave the generated copyright header that includes the LLM's source material. It' not hard to imagine that more selfish people tweak the code to hide the origin. The same situation as the AI-generated homework essays.
I generally like AI coding using CC etc, but this forced me to remember that these generated code ultimately came from these stolen (spiritually, not necessarily legally) pieces.
Did these Ocaml maintainers undergo some special course for dealing with difficult people? They show enormous amounts of maturity and patience. I'd just give the offender Torvalds' treatment and block them from the repo, case closed.
I wonder if it's the best outcome? The contributor doesn't seem to have a bad intention, could his energy be redirected more constructively? E.g. encouraging him to split up the PR, make a design proposal etc.
The constructive outcome is the spammer fucks off or learns how to actually code.
Lots of people all over the world learn some basics of music in school, or even learn how to play the recorder, but if you mail The Rolling Stones with your "suggestions" you aren't going to get a response and certainly not a response that encourages you to keep spamming them with "better" recommendations.
The maintainers of an open source project are perfectly capable of coercing an LLM into generating code. You add nothing by submitting AI created code that you don't even understand. The very thought that you are somehow contributing is the highest level of hubris and ego.
No, there's is nothing you can submit without understanding code that they could not equally generate or write, and no, you do not have an idea so immensely valuable that it's necessary to vibe code a version.
If you CAN understand code, write and submit a PR the standard way. If you cannot understand code, you are wasting everyone's time because you are selfish.
This goes for LLM generated code in companies as well. If it's not clear and obvious from the PR that you went through and engineered the code generated, fixed up the wrong assumptions, cleaned up places where the LLM wasn't given tight enough specs, etc, then your code is not worth spending any time reviewing.
I can prompt Claude myself thank you.
The primary problem with these tools is that assholes are so utterly convinced that their time is infinitely valuable and my time is valueless because these people have stupidly overinflated egos. They believe their trash, unworkable, unmaintainable slop puked out by an LLM is so damn valuable, because that's just how smart they are.
Imagine going up to the Civil Engineer building a bridge and handing them a printout from ChatGPT when you asked it "How do you build a bridge" and feeling smug and successful. That's what this is.
I believe there's a flood of people waiting to be able to "contribute" by publishing a lot of LLM generated code. My question is what if they manage to grab resources from the original devs ?
I think it's for me to redo the PR and break it into smaller pieces.
There's value in the PR in that it does not require you to install the separate OxCaml fork from Jane St which doesn't work with all the OCaml packages. Or wasn't when I tried it back in August.
In my big tech company, you don't want to be dismissive of AI if you don't want to sound like a paria. It's hard to believe how much faith leadership has in AI. They really want every engineer to use AI as much as possible. Reviewing is increasingly done by AI as well.
That being said, I don't think that's why reviewers here were so cordial, but this is the tone you'd expect in the corporate world.
To all the AI apologists here I'd like to submit a simple scenario to you and hear your answer: you use AI to create a keynote speech on a topic you needed to use AI to write. At the end of your speech, people ask you questions about the contents of your speech. What do you say?
Hi, AI apologist here. This scenario is a problem with or without AI. You can’t drop a 13k line PR you don’t understand without prior discussion. There are many ways to use AI. Your scenario (keynote speech) is a bad way to use it. Instead, a PR where you understand every line, whether you or an AI wrote it, should be fine. It would be indistinguishable from human generated code.
AI is a tool like any other. I hire a carpenter who knows how to build furniture. Whether he uses a Japanese pullsaw or a CNC machine is irrelevant to me.
88 comments
[ 2.5 ms ] story [ 99.1 ms ] threadI guess the proponents are right. We'll use LLMs one way or another, after all. They'll become one.
Discussions about AI/LLM code being a problem solely because AI/LLM is not generally a productive conversation.
Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc.
Additionally, if there isn't a code of conduct, AI policy, or, perhaps most importantly, a policy on how to submit PRs and which are acceptable, it's a huge weakness in a project.
In this case, clearly some feathers were ruffled but cool heads prevailed. Well done in the end..
>>> Beats me. AI decided to do so and I didn't question it.
Really sums the whole thing up...
One of the biggest problems is the fact that the public nature of Github means that fixes are worth "Faux Internet Points" and a bunch of doofuses at companies like Google made "social contribution" part of the dumbass employee evaluation process.
Forcing a person to sign up would at least stop people who need "Faux Internet Points" from doing a drive-by.
No centralisation of my code in siloes like Github, I won't have to care about bots making hundreds of requests on my self-hosted Gitea instance, would prove to be a noticeable source of friction to vibe coders, and I don't care about receiving tons of external contributions from whomever.
For serious people, it'll only be a matter of running `git format-patch` and sending me an attachment via email.
(Not so)interestingly, the PR author even advertised this work on HN.
Oh, wow. They're being way too tolerant IMO; I'd have just blocked him from the repo at about that point.
Found this part hilarious - git ignoring all of the claude planning MD files that it tends to spit out, and including that in the PR
Lazy AI-driven contributions like this are why so many open source maintainers have a negative reaction to any AI-generated code
Repeatedly using AI to answer questions about the legitimacy of commits from an AI, to people who are clearly skeptical is breathtakingly dense. At least they're open about it.
I did love the ~"I'll help maintain this trash mountain, but I'll need paying". Classy.
> Damn, I can’t debug OCaml on my Mac because there’s no DWARF info…But, hey, there’s AI and it seems to one-shot fairly complex stuff in different languages, from just a Github issue…My needs are finally taken care of!
So I do believe using an LLM to generate a big feature like OP did can be very useful, so much that I’m expecting to see such cases more frequently soon. Perhaps in the future, everyone will be constantly generating big program/library extensions that are buggy except for their particular usecase, could be swapped with someone else’s non-public extensions that they generated for the same usecase, and must be re-generated each time the main program/library updates. And that’s OK, as long as the code generation doesn’t use too much energy or cause unforeseen problems. Even badly-written code is still useful when it works.
What’s probably not useful is submitting such code as a PR. Even if it works for its original use-case, it almost certainly still has bugs, and even ignoring bugs it adds tech debt (with bugs, the tech debt is significantly worse). Our code already depends on enough libraries that are complicated, buggy, and badly-written, to the extent that they slow development and make some feasible-sounding features infeasible; let’s not make it worse.
I wonder how long the open-source ecosystem will be able to resist this wave. The burden of reviewing AI-generated PRs is already not sustainable for maintainers, and the number of real open-source contributors is decreasing.
Side note: discovering the discussions in this PR is exactly why I love HN. It's like witnessing the changes in our trade in real time.
> in my personal experience, reviewing AI-written code is more taxing that reviewing human-written code
I think the burden is on AI fanbois to ship amazing tools in novel projects before they ask projects with reputations to risk it all on their hype.
To deliver a kernel of truth wrapped in a big bale of sarcasm: you're thinking of it all wrong! The maintainers are supposed to also use AI tools to review the PRs. That's much more sustainable and would allow them to merge 13,000 line PRs several times a day, instead of taking weeks/months to discuss every little feature.
The difference here of course is in how impressed you are by AI tools. The OCaml maintainers are not (and rightly so, IMO), whereas the PR submitter thinks they're so totally awesome and leaving tons of productivity on the table because they're scared of progress or insecure about their jobs or whatever.
Maybe OCaml could advance rapidly if they just YOLO merged big ambitious AI generated PRs (after doing AI code reviews) but that would be a high risk move. They have a reputation for being mature, high quality, and (insanely) reasonable. They would torch it very quickly if people knew this was happening and I think most people here would say the results would be predictably bad.
But lets take the submitter's argument at face value. If AI is so awesome, then we should be able to ship code in new projects unhampered by gatekeepers who insist on keeping slow humans in the loop. Or, to paraphrase other AI skeptics, where's all of the shovelware? How come all of these AI fanbois can only think about laundering their contributions through mature projects instead of cranking out amazing new stuff?
Where's my OCaml compiler 100% re-written in Rust that only depends on the Linux kernel ABI? Should cost a few hundred bucks in Claude credits at most?
To be clear, the submitter has gotten the point and said he was taking his scraps and going to make his own sausage (some Lisp thing). The outcome of that project should be very informative.
in response to someone asking about why the author name doesn't match the contributor's name. Incredible response.
I generally like AI coding using CC etc, but this forced me to remember that these generated code ultimately came from these stolen (spiritually, not necessarily legally) pieces.
Lots of people all over the world learn some basics of music in school, or even learn how to play the recorder, but if you mail The Rolling Stones with your "suggestions" you aren't going to get a response and certainly not a response that encourages you to keep spamming them with "better" recommendations.
The maintainers of an open source project are perfectly capable of coercing an LLM into generating code. You add nothing by submitting AI created code that you don't even understand. The very thought that you are somehow contributing is the highest level of hubris and ego.
No, there's is nothing you can submit without understanding code that they could not equally generate or write, and no, you do not have an idea so immensely valuable that it's necessary to vibe code a version.
If you CAN understand code, write and submit a PR the standard way. If you cannot understand code, you are wasting everyone's time because you are selfish.
This goes for LLM generated code in companies as well. If it's not clear and obvious from the PR that you went through and engineered the code generated, fixed up the wrong assumptions, cleaned up places where the LLM wasn't given tight enough specs, etc, then your code is not worth spending any time reviewing.
I can prompt Claude myself thank you.
The primary problem with these tools is that assholes are so utterly convinced that their time is infinitely valuable and my time is valueless because these people have stupidly overinflated egos. They believe their trash, unworkable, unmaintainable slop puked out by an LLM is so damn valuable, because that's just how smart they are.
Imagine going up to the Civil Engineer building a bridge and handing them a printout from ChatGPT when you asked it "How do you build a bridge" and feeling smug and successful. That's what this is.
There's value in the PR in that it does not require you to install the separate OxCaml fork from Jane St which doesn't work with all the OCaml packages. Or wasn't when I tried it back in August.
That being said, I don't think that's why reviewers here were so cordial, but this is the tone you'd expect in the corporate world.
This is the same.
AI is a tool like any other. I hire a carpenter who knows how to build furniture. Whether he uses a Japanese pullsaw or a CNC machine is irrelevant to me.
> Different naming conventions (DW_OP_* vs DW_op_*)