> The changes were harmless and correct, but that did not make me feel better about accepting or merging them.
So do you have the project’s best interest at heart or not? If you’re more concerned about the intent of a valid contribution than the content, why don’t you ask yourself where your intent is? You rejected a valid contribution based on unverified vibes about the person’s intent, instead of assuming they were just being helpful.
Why not assume someone does have the best interest of their own project at heart? How do you get to question that out of the gate, while pretending automated grammar and spelling fixes to rack up PR counts are benevolent unless proven otherwise? Simply assume the person who made the thing you didn't make, who was the only person (or persons) on the planet to come up with that exact thing, knows what's best for that person (or persons). Just like you would handle your own stuff.
I think it’s implied that accepting these kinds of PRs will take time away from working on actual improvements to the codebase. So, on the long term it’s better to be strict with these kinds of silly contributions, and focus on changes that matter more (e.g. features or fixes).
that won't work because all those millions of 'projects' are unmaintainable vaporware which is exactly why people try to piggyback on the reputation of software that is actually being used and developed by people
Hi Neil, fun to see you on HN. I agree with your points and I you summarized it very well as "Ultimately, open source is built on trust".
AI is destroying trust in open source and many other areas and I think this will discourage teams from publishing their source code in the future.
On the other hand, personal connections are becoming even more important, which is unfair to the younger generation and people who don't live near tech hubs.
I was thinking the same. The amount of effort people went through just for a t-shirt. Now you can try to boost your career, far more valuable, and even with less effort. Of course it will happen, unfortunately.
This makes sense. Apart from the original thing, I no longer upstream anything. It just takes comparatively more effort than maintaining a private fork with my own idiosyncratic fix. This must mean that even small fixes that non-contributors would make in the past are just happening off the books, so to speak. e.g. I made a tiny fix to Thrift codegen because some function could be much faster. These days I wouldn't bother. I'd just fork and leave upstream to be upstream.
I suspect many people are like me since it feels like a very normie position to take. That means that contributors are even more likely to be useful because both the drive-by genuine contributors have chosen otherwise and these contributors have increased.
It must be quite painful to be an open source maintainer right now. One wonders what to make of such projects in a future where a feature-list is a sufficient prompt.
I ran through exactly this thought process & LLM based resolution just 8 hours ago with a segfault compiling a project which likely doesn't care about the given platform combo much.
Classic problem. I recall spending a long time trying to get Freespace 2 SCP working on the Via Unichrome onboard GPU I had decades ago. Who could possibly have cared for that? The intersection of the two sets was probably precisely one person.
By the way, interesting person to have 512 IPv4s (and two /44s? Aren't those enormous?). Root-level http intentionally user/pass gated on your site? I was curious to see.
:D. I've spent most of my career in networking or network adjacent. I wanted to have my own IP space on the internet over BGP peering from 2 locations for redundancy. Two /23s and a /48 actually end up being the pretty minimalistic take on going about it. The biggest block I've ever had the privilege of requesting/deploying was for an extremely large healthcare organization who got a /32... of IPv6!
As for why so many IPs because of that: The smallest IPv6 segment you can advertise on the open internet is a /48, but to get two of those to advertise from the same block you need a /44 since assignments are (currently) done on the nibble boundary. I put this request in and the RIR recommended a /40 so I could have one /44 pool per site. It was the same renewal fee, and made a lot of sense from the advertisement failover simplifcation so I went for it.
Since I didn't have any IPv4 but I also host the DNS server for the domain, that qualified me for a "free" (still pay normal registration renewals of course) lease of a /24 IPv4 block (256 addresses) per site to dual stack these core services (/24 being the smallest IPv4 unit you can advertise on the internet). It irked me a bit they did this as two separate /24s instead of a /23 since it means the route advertisement failover isn't as simple as advertising the /23 and local /24 at the same time like with the IPv6 breakout (i.e. /23s are accepted on the internet but when TPKI goes to check what resources I have it will say I only have rights to advertise the two /24s, not the parent /23).
All in all, it was something around ~$1k to go from no registration to org id + ASN + IPv4 + IPv6.
As for the site, I really should get around to making a public portion some time. Maybe a good December break project this year :).
Having seen this happening in the past, pre-AI, the answer is usually that they don't. They just stay on that version forever. Or it's maintained until the original author moves on, after which it's someone else's problem to figure out how to move back to the upstream, which has now had years of changes.
Discussing the issues, achieving consensus, making sure the fix fits the existing patterns and actually warrants the change, etc. These are and have always been a challenging part of contributing to open source. Typing code in your private fork that satisfies your “idiosyncratic fix” has always existed and been always the easier aspect of such changes. Chances are many of these changes would never be accepted into the repository anyway in the past.
Certainly, but there was motivation to do that in the past. My patch would probably line up with the rest of the code in the direction that they desired and the cost of fork maintenance would have pushed me to upstream in a way acceptable to maintainers.
I find it hard to believe that this is the main motivation for most who contributors to OSS. Do you have a reference, research or poll that corroborate this?
You’re vastly overestimating the value of these “fixes” that now go to private forks, because maintainers also have access to LLMs and can prompt, probably better than a drive by fix since they have better understanding and context of the codebase.
Therefore I believe, in most cases, keeping those fixes to private forks is not a noticeable loss for the OSS projects.
Fair enough not to want to get involved but seems problematic for 2 reasons :
- code gets re-regenerated over and over again, so waste of resources
- no signal back upstream that it is indeed a genuine need that is in fact so important some users did dedicate resources for it, maybe more users would benefit from it
All this is true but I prefer writing AI assisted code and I doubt any maintainer can distinguish me from full spectrum bot. It’s just not productive to either of us.
So the fixes are still fixes, but we (I am also a OSS maintainer) are unwilling to accept them as they boost the contributor’s status where we think the merit is very or extremely limited.
Why not have these PRs counted differently (by the platform), and/or colored differently in the timeline(s) thus made less visible or more clear?
Unless the change is an obvious improvement, it has to be worth the time for the maintainers to spend attention reviewing it (and supporting the code forever, and all the rest).
Even if these particular changes are "harmless" and easy to review, accepting them sets a precedent that encourages an unsustainable floodgate of AI-generated changes that will overwhelm the project.
That’s a totally unnecessary high bar you’re setting. Most change is mediocre (a tautology, I realize). I really don’t believe that you can think that fixing a bunch of spelling mistakes is not a small improvement.
About setting a precedent: if you think like that you probably never accept any contributions at all, by humans or otherwise. Contributions by strangers have always been very minor things in general, things that the author cared enough about to make a pull request. In this case I don’t see the difference except for the fact that ai was used. If you just don’t accept ai at all , fine. But just be clear about it. In this case they are calling spelling correction slop. That’s not what slop means. Pretending human contributions were always “great” and nothing else would do is kind of ridiculous and shows a lack of experience in open source.
Change is bad. It takes engineers time to understand and approve. It confuses users. Changes should be made for good reasons, not frivolously.
Just because writing code is easier does not mean you should write more of it. It is more important than ever to carefully consider what you work on and what work you accept.
The user says "this is just spelling corrections" but a maintainer - a volunteer - needs to read all the files and make sure it's harmless and doesn't introduce security vulnerabilities, change meanings, and so on. That's not free.
Sorry, this is a tangent and generally I agree with your first comment, but I don't know when else I'll be able to share this.
We once had a new engineer run a spell-checker across our entire codebase on his first week on our team.
He opened a PR that was in the ballpark of 10k lines changed, including references to 3rd party API calls where we were making requests to valid endpoints with gramatical oddities like /userses that his spell-checker had fixed due to a reliance on spelling correctness rather than interface interop.
I was a little more junior than him, but it was my project. I think it took me 2 days to convince him it was a net negative.
I tend not to change spelling errors willy-nilly due to that PR.
Surprisingly prompting takes skill. The more you know about something the more precise you can be and the better AI actually works. I’ve found just saying “here is my bug/issue, fix this” is about the absolute worst way thing you could do, but give it a clear direction and precise step by step instructions and it will not only do well but sometimes blow your mind.
Because every merge request and every merge comes with cost and (potential) more cost in the future.
Linux lately received a lot of fixes for drivers nobody has cared about for ages. They decided to remove them. Maybe they should have done this earlier, I don't know, but that happened because every code change comes with cost down the line. For real contributers, this cost is much more limited and they have a higher probability of burden the cost down the line.
Pure AI MRs are just rude, because it's just "here take this, don't really care what this is, but AI said it being good, I won't be around, so you will have to deal with this code after the merge and you better don't overlook anything, because I personally don't really know what I am doing, I only know how to have AI making something that looks convincing. Good luck fixing the bugs this change introduces".
Because letting sloperators near your codebase, even for a very minor obvious and correct change, may have long-term negative consequences for your project. It sends a signal to other sloperators to keep doing what they do, when we really need to send them a signal to get a life, learn a craft, and let Claude burn in hell.
I think what you're saying is fair, tbh. Imagine you put in months or years into a project to make it a quality piece of work. It develops into something notable and you took all the risk. Then someone comes along to fix a spelling error with a pull request so their name effectively appears on the repo as a "contributor." And you just know right after its going on their resume as "contributed to [...]" or maybe if they're bold "software engineer working on [...]" which implies substantial investment. Then you're effectively sharing credit for YOUR work with someone who did nothing. That is rage inducing. ((Of course: it probably is just juniors trying their best in this horrible industry.))
On the other hand: lets be careful not to dismiss valid but inexperienced attempts to contribute. Having someone want to genuinely contribute to your software is incredibly generous. If someone seems like they're trying its better to give advice than act like a snob because its not good enough. Often pull requests only need small fixes to get in, anyway.
These fixes, regularly destroy the architecture, accrue bloat for little gain, refuse to rewrite while demanding to rewrite- and many other such funny noises. Most code contribution by LLMs is garbage if you long-term care about the project. Look at closed source projects that ingest all this madness - windows with its seconds to open the explorer and other catastrophes.
maybe, the maintainer would need to review them to determine that, and that can be time-consuming especially because LLMs tend to provide walls of sometimes dense text
since the human submitting the PR (if there is one) couldn't be bothered to understand whether the change is a valid one (and therefore submit the change as their own work), why should the maintainer take on that work. if they wanted an LLM to find bugs in their code they could run it themselves with less work than sorting through LLM-generated PRs.
I very much can relate, as I've also been receiving many of these drive-by PRs lately.
One big problem I have with them is that they take away time from project maintainers for reviewing and helping to get the PRs into shape, which then can't be spent on other, more important things. I feel like the "good first issue" GitHub label is specifically attracting these kinds of contributions.
It's not a black-or-white thing though, and you need to tell apart folks who produce slop PRs against any arbitrary repo, from folks using AI to contribute in a sensible way. We've tried to codify some rules in our contribution guide [1]:
- PRs from apparent bot accounts are closed
- PRs from users who file large numbers against random repos are closed
- You're welcome to use AI, but you need to stand behind your PR and be able to explain it
I'm sure we'll adjust those rules over time, but since we have instantiated them, it definitely has become easier to deal with AI PRs and handle them in an a relatively objective way. It absolutely means that sometimes a PR will be closed which could have been an improvement, but I think this is the right thing to do given the circumstances.
AI code review is nice to have an additional pair of eyes but it doesn't substitute to the maintainer eyes.
A big part of a review is deciding if you want that change merged in. Not because of the immediate code but what it means to the project to bring this in.
AI code review, to my eyes, just reduce the number of bugs, they do not shorten the review time. Otherwise, it means you are compromising on the direction of the project.
I help maintain a big project as well and I can totally relates what the author is describing.
The amount of security advisories and PRs opened is getting out of hands.
This is no longer just spell check PRs that you can merge in less than 1 minute of review. Often that's going to be supposed perf improvement, attempts at fixing things that are not broken, or submitting completely new features.
These relatively low effort PRs takes a lot of effort to review or even just to filter out.
GitHub flavored FOSS I believe works best for "corporate FOSS" projects and terrible for "FOSS as how it was conceived decades ago".
Or rather I believe that it is engineered exactly for the former.
What it does well is coordination between corps, working in public (while on a corp payroll) and extracting some drive-by-PR value out of random third-parties.
The closer your project is to that shape, the better it works for you. The further it is away, the more pain you will feel.
This has nothing to do with AI slop.
AI slop just turned up a few knobs that were already there.
I also maintain a popular open source repo and have seen similar. If I receive a low effort single-pass obviously-Clauded PR (many people (agents) don’t even bother using the AGENTS.md file I provide right in the repo?!) with no associated issue I also have no qualms closing. I do generally leave a short note about why I closed (volume of these is around 5/week) showing how far off they (Claude) was and encourage opening an issue where we can start a discussion which rarely happens for the drive-by folks. So I totally feel this but I find myself more disheartened by the practice than angry. It feels like slowly watching open-source go the way of email. Open and free until no-cost spam ruined the inbox for everyone...
Do recruiters actually care about your open source contributions? Especially now when only LLMs read CVs and match them against strict criteria, I doubt there are many companies that actually care about your work outside of work, in fact they might care in opposite direction - thinking that you'll be distracted from work
Ironically this seems like a perfect use for AI from the maintaner side. "Find PR's without associated issues from new contributors. Discuss with the contributors in the PR about how contributions should be made, then reject their PR's. If any bad behavior is detected in this interaction, block the accounts for 24 hours and add them to a list of accounts to review for permanent banning from the project".
Automated PR's can have automated responses. You make a human effort, you get a human response.
We do similar in Homebrew by autoclosing opens without the issue template (which implies using the API which often now implies AI).
The weird flip side is that the average good AI contribution is better than the average “I used no AI” contribution. Perhaps it’s because Homebrew has so many declarative guardrails and is so easy for agents to test.
As long as you have it disclose itself: feel free. We're not anti-AI use, we're just anti-low-effort AI use and don't see the need to apply effort to educate people who haven't applied the effort themselves.
I think the idea of leveraging AI to find useless PRs is right.
I didn’t like the second part though where it starts banning people for “bad behavior”.
I get that AI resistance will be a thing in the next few years, but realistically engineers who plan on being around for the next 20-30 years just need to embrace it instead of being sour about it.
Because of machines, things have arguably improved for humanity as a whole. People who seek more elevated/finer products can still turn to handcrafted products.
Mass software will be produced by AI, and it will scale just fine.
> I didn’t like the second part though where it starts banning people for “bad behavior”.
I think you failed to notice the ban would be a reaction to recurrent bad behavior, such as ignoring maintainer requests.
What's your solution to handle accounts which spam projects with nonsense PRs?
> get that AI resistance will be a thing in the next few years (...)
This is a very lazy opinion to express. If you pay attention to the post you're replying to, you will notice that the problem isn't AI. The problem is spamming projects with PRs that are meaningless while completely ignoring maintainers.
You'd have the same problem if some guy decided to spam the project with a constant barrage of small low-effort nonfunctional PRs.
You should really read the comments you are replying to. The whole point of the proposal is to provide actionable feedback to the people operating these drive-by PR bots, and kick out repeat offenders to prevent them from generating noise.
> What will be happening is that bot will happily talk with your but, but falsely flagged people will be antagonized and eventually angry.
I think you're trying too hard to come up with imaginary scenarios. Also, keep in mind that the objective is to steer contributions to be aligned with the project goals and help maintainers not waste time with low-quality noise. The goal is not to impose a fundamentalist militant stance against AI.
I did read it. It is not a plan to kick repeat offendrs. It is not a plan to give actionable feedback.
This plan will kick no bots, because bots dont mind discussion with another bot. However, falsely flagged people will realize they are talking with bot prompted to be obtuse, get annoyed anf then will be banned
> the objective is to steer contributions to be aligned with the project goals
Which part of the second part do you dislike more?
That AI will be given the power to ban people?
Or that AI will be the ones getting banned?
If it's the former, expect an invite from the AI resistance. :)
If it's the latter, well, that can be fixed by spending more tokens and writing better PRs so that the AI moderator doesn't think they're "low-effort, AI-like". Low-effort is low-effort, whether it's from humans or AI.
> Because of machines, things have arguably improved for humanity as a whole.
That's a very narrow view of the situation. I agree that this side of the coin exists, but there's also the other side where drinkable tap water is becoming a luxury in much of the global North, giant tornadoes and fires are sweeping the Earth, and temperatures are growing out of control.
So, to make lives more comfortable with machines for some millions of privileged people, they triggered the 6th mass extinction. That's the other side of the coin.
We're already at the end of the 6th mass extinction FWIW. Over the last 200 years most species have gone extinct. There isn't that much left to make extinct. We'll manage it though.
Can you say more about this? The first link below [1] says there are an estimated eight million species today, and the second link [2] says the number of known extinct species since 1500 is 905.
As a contributor who had to deal with 2 automated reviews in the past month, it's a bad experience. I really wanted to upstream my changes, but these bots made me question if i should at all.
Bots don't understand how their words can hurt people's feelings, and can even less understand that talking to a LLM is even more sad/infuriating for actual humans. I can handle CI/tests just fine, but LLMs playing on my human emotions will have me swearing in no time.
I contribute to free software projects because i want user control over the machine (that was the idea, long before LLMs). Slopware might technically be FLOSS, but it's certainly not in the spirit of FLOSS.
I haven't been exposed AI reviews in OSS but I like them for work. Quality of PR review feedback is the most revolutionary bit of AI I think. More so than automated coding. I guess one issue with the AI reviews in OSS is that your dialog (I assume) is public. You know the interaction is with a bot but you also sense it may be observed by humans.
If I get a review at work from a bot that is annoying, misunderstands, etc I just ignore their reviews, or tell the bot to leave me alone. The bot is there to help me, and if it doesn't - that's not my problem. There's also always a human reviewer. For any automated feedback for PR's I think there should be "manual review" requests. If you disagree and the bot can decide that there is real human effort involved, then just let real humans also make the effort to review.
This is the key idea: to spend the human effort from maintainers, where there has been human effort from contributors. The AI should be able to determine the amount of human effort in a contribution. The bot should be an aid to make sure the human maintainer effort is spent only where there is human contributor effort.
> Bots don't understand how their words can hurt people's feeling
That's a super weird way of putting it, which makes me hesitate to agree with you.
buuuut I agree that the LLM bot review experience is usually miserable, as the persona these things use is insufferable. It's a know-it-all only-speaks-in-imperatives thing, where the speaker lacks the actual merit that would grant them the right to that style.
LLMs can be very useful for code review, but the existing product implementations took a bit too many pages out of Kafka's works and are a bit too real in simulating a dysfunctional bureaucratic org.
> That's a super weird way of putting it, which makes me hesitate to agree with you.
Yes that's weird sorry. What i meant is it belongs in the uncanny valley. I don't have any good/bad feelings about test suites and CI feedback, it's very binary. Receiving a mail about a comment on my PR raises my hopes that it reached the maintainers (or interest from other contributors); receiving LLM text destroys my expectations.
Then, about the content itself, it has a very patronizing tone. Saying out loud every single thing that's in my +25/-3 PR is really infantilizing. I would probably have slightly less negative reactions if it only commented to suggest improvements, instead of patting me on the back. See this example: https://github.com/TwiN/gatus/pull/1712#pullrequestreview-46...
Seems like a contributor reputation score (shared across all projects) that one can build up over time would be a clean solution to the low quality pull requests.
I think opensource has decided to become big corporate cheerleader for a while and not actually done things strictly in the interest of users or developers, but providing the ideological justification as to why big companies can use their code and not give a dime back. I think aside from free software there could have been a movement where contributing to a free-software project would have allowed contributors to get payed and this would have to preclude companies simply using code as they can in say the MIT license and not give back.
Instead opensource is full of cynical projects and real important projects that do amazing work with maintainers who shoulder a great burden having to listen to people bitch about them for features and fixes while giving nothing and dealing with bad PR's (or even just having to evaluate good and mediocre PR's). Often the only pay is that you can go to HR and shit corp with your CV and say "Here is what I have done, can I now be treated like a bitch for more money".
I think opensource has allowed a lot businesses to treat developers with contempt and this is fully manifest with the way they conceive of AI agent's and Open source has probably become a vector for demoralization and devaluation more then anything.
IMHO, the days when open source contributions are often positive signal for hiring are long gone.
Actually, if I see someone doing open source like it's a performative career checkbox, that will not be positive signal, and could easily be negative. I understand that people will do what they need to do to get a job, but that pragmatic career checkboxing itself isn't positive. I will have to look for positive signal elsewhere for that person.
The problem is that our industry has gotten very bad at hiring, and now we have everyone playing all sorts of games with it -- rehearsing Leetcode interviews, bad faith open source contributions, feigning enthusiasm, spamming AI-tweaked resumes, outright cheating on interviews -- rather than focusing on doing good work, and being part of a team.
If you're involved in hiring, and you care about effectiveness and culture, consider pushing back against the prevailing dysfunctional big-corporate-cattle-herding practices. Especially if your company has no excuse to be big-corporate dysfunctional, and can't afford to be.
> IMHO, the days when open source contributions are often positive signal for hiring are long gone.
After reading your comment, I had a thought that the key could be to separate random, drive-by open source contributions (which are easily gamed by pointing a bunch of AI agents at GitHub) from committing to maintain a small number of open source/community projects for the long term.
Unfortunately, I think the appearance of long-term maintaining of an open source project would be just another gameable metric, especially with delegation to an AI slave, to make it almost zero cost.
Helicopter parents and then young adults are already long-term gaming metrics, before they start high school, just to get into college, and it seems that the same kinds of thinking persist through first job and rest of career.
Being seen as a long-term maintainer of an open source project is now trivial, compared to some of the other things people already do.
Offhand, ideas:
1. Use criteria the metrics-gamers won't think of, and don't tell them what it is. (Why this might work: the metrics-gaming mindset might blind people to thinking outside that mindset. Why this might not work: you need someone not blinded to apply the criteria.)
2. Do things that are unattractive to the metrics-gamers, but attractive to people who aren't. (Though you might still get overwhelmed by near-zero-cost AI spamming applications anyway.)
3. Accept that too many current workers were brought up with metrics-gaming mindset, and the others are too hard to find in the noise, and select and align metrics-gamers in game-theoretic ways. (Extreme example: you pay minimum wage, and there are no promotions nor titles nor performance evaluation metrics not anything except success of the company, but everyone gets real equity that's worth zero unless the team together delivers and wins, in which case everyone gets equally wealthy. The trick would be convincing the dumber metrics-gamers that they can't just nod and game this, and if they don't play like a team member for the team to win, they will be voted off the island and get zero, and possibly left in a ditch.)
105 comments
[ 0.19 ms ] story [ 12.3 ms ] threadSo do you have the project’s best interest at heart or not? If you’re more concerned about the intent of a valid contribution than the content, why don’t you ask yourself where your intent is? You rejected a valid contribution based on unverified vibes about the person’s intent, instead of assuming they were just being helpful.
Obviously the best way to use AI to furnish your CV is to build your own project using AI.
AI is destroying trust in open source and many other areas and I think this will discourage teams from publishing their source code in the future.
On the other hand, personal connections are becoming even more important, which is unfair to the younger generation and people who don't live near tech hubs.
I suspect many people are like me since it feels like a very normie position to take. That means that contributors are even more likely to be useful because both the drive-by genuine contributors have chosen otherwise and these contributors have increased.
It must be quite painful to be an open source maintainer right now. One wonders what to make of such projects in a future where a feature-list is a sufficient prompt.
By the way, interesting person to have 512 IPv4s (and two /44s? Aren't those enormous?). Root-level http intentionally user/pass gated on your site? I was curious to see.
As for why so many IPs because of that: The smallest IPv6 segment you can advertise on the open internet is a /48, but to get two of those to advertise from the same block you need a /44 since assignments are (currently) done on the nibble boundary. I put this request in and the RIR recommended a /40 so I could have one /44 pool per site. It was the same renewal fee, and made a lot of sense from the advertisement failover simplifcation so I went for it.
Since I didn't have any IPv4 but I also host the DNS server for the domain, that qualified me for a "free" (still pay normal registration renewals of course) lease of a /24 IPv4 block (256 addresses) per site to dual stack these core services (/24 being the smallest IPv4 unit you can advertise on the internet). It irked me a bit they did this as two separate /24s instead of a /23 since it means the route advertisement failover isn't as simple as advertising the /23 and local /24 at the same time like with the IPv6 breakout (i.e. /23s are accepted on the internet but when TPKI goes to check what resources I have it will say I only have rights to advertise the two /24s, not the parent /23).
All in all, it was something around ~$1k to go from no registration to org id + ASN + IPv4 + IPv6.
As for the site, I really should get around to making a public portion some time. Maybe a good December break project this year :).
You’re vastly overestimating the value of these “fixes” that now go to private forks, because maintainers also have access to LLMs and can prompt, probably better than a drive by fix since they have better understanding and context of the codebase.
Therefore I believe, in most cases, keeping those fixes to private forks is not a noticeable loss for the OSS projects.
- code gets re-regenerated over and over again, so waste of resources
- no signal back upstream that it is indeed a genuine need that is in fact so important some users did dedicate resources for it, maybe more users would benefit from it
Why not have these PRs counted differently (by the platform), and/or colored differently in the timeline(s) thus made less visible or more clear?
Unless the change is an obvious improvement, it has to be worth the time for the maintainers to spend attention reviewing it (and supporting the code forever, and all the rest).
Even if these particular changes are "harmless" and easy to review, accepting them sets a precedent that encourages an unsustainable floodgate of AI-generated changes that will overwhelm the project.
About setting a precedent: if you think like that you probably never accept any contributions at all, by humans or otherwise. Contributions by strangers have always been very minor things in general, things that the author cared enough about to make a pull request. In this case I don’t see the difference except for the fact that ai was used. If you just don’t accept ai at all , fine. But just be clear about it. In this case they are calling spelling correction slop. That’s not what slop means. Pretending human contributions were always “great” and nothing else would do is kind of ridiculous and shows a lack of experience in open source.
Change is bad. It takes engineers time to understand and approve. It confuses users. Changes should be made for good reasons, not frivolously.
Just because writing code is easier does not mean you should write more of it. It is more important than ever to carefully consider what you work on and what work you accept.
The user says "this is just spelling corrections" but a maintainer - a volunteer - needs to read all the files and make sure it's harmless and doesn't introduce security vulnerabilities, change meanings, and so on. That's not free.
We once had a new engineer run a spell-checker across our entire codebase on his first week on our team.
He opened a PR that was in the ballpark of 10k lines changed, including references to 3rd party API calls where we were making requests to valid endpoints with gramatical oddities like /userses that his spell-checker had fixed due to a reliance on spelling correctness rather than interface interop.
I was a little more junior than him, but it was my project. I think it took me 2 days to convince him it was a net negative.
I tend not to change spelling errors willy-nilly due to that PR.
How do you distinguish those who know what they are doing from those who only can write a prompt and copy the result?
Making the haystack bigger is a bad idea if you search the needle
Linux lately received a lot of fixes for drivers nobody has cared about for ages. They decided to remove them. Maybe they should have done this earlier, I don't know, but that happened because every code change comes with cost down the line. For real contributers, this cost is much more limited and they have a higher probability of burden the cost down the line.
Pure AI MRs are just rude, because it's just "here take this, don't really care what this is, but AI said it being good, I won't be around, so you will have to deal with this code after the merge and you better don't overlook anything, because I personally don't really know what I am doing, I only know how to have AI making something that looks convincing. Good luck fixing the bugs this change introduces".
On the other hand: lets be careful not to dismiss valid but inexperienced attempts to contribute. Having someone want to genuinely contribute to your software is incredibly generous. If someone seems like they're trying its better to give advice than act like a snob because its not good enough. Often pull requests only need small fixes to get in, anyway.
maybe, the maintainer would need to review them to determine that, and that can be time-consuming especially because LLMs tend to provide walls of sometimes dense text
since the human submitting the PR (if there is one) couldn't be bothered to understand whether the change is a valid one (and therefore submit the change as their own work), why should the maintainer take on that work. if they wanted an LLM to find bugs in their code they could run it themselves with less work than sorting through LLM-generated PRs.
One big problem I have with them is that they take away time from project maintainers for reviewing and helping to get the PRs into shape, which then can't be spent on other, more important things. I feel like the "good first issue" GitHub label is specifically attracting these kinds of contributions.
It's not a black-or-white thing though, and you need to tell apart folks who produce slop PRs against any arbitrary repo, from folks using AI to contribute in a sensible way. We've tried to codify some rules in our contribution guide [1]:
- PRs from apparent bot accounts are closed - PRs from users who file large numbers against random repos are closed - You're welcome to use AI, but you need to stand behind your PR and be able to explain it
I'm sure we'll adjust those rules over time, but since we have instantiated them, it definitely has become easier to deal with AI PRs and handle them in an a relatively objective way. It absolutely means that sometimes a PR will be closed which could have been an improvement, but I think this is the right thing to do given the circumstances.
[1] https://github.com/hardwood-hq/hardwood/blob/main/CONTRIBUTI...
AI code review, to my eyes, just reduce the number of bugs, they do not shorten the review time. Otherwise, it means you are compromising on the direction of the project.
The amount of security advisories and PRs opened is getting out of hands. This is no longer just spell check PRs that you can merge in less than 1 minute of review. Often that's going to be supposed perf improvement, attempts at fixing things that are not broken, or submitting completely new features.
These relatively low effort PRs takes a lot of effort to review or even just to filter out.
What it does well is coordination between corps, working in public (while on a corp payroll) and extracting some drive-by-PR value out of random third-parties.
The closer your project is to that shape, the better it works for you. The further it is away, the more pain you will feel.
This has nothing to do with AI slop. AI slop just turned up a few knobs that were already there.
https://en.wikipedia.org/wiki/The_Sorcerer%27s_Apprentice
Automated PR's can have automated responses. You make a human effort, you get a human response.
The weird flip side is that the average good AI contribution is better than the average “I used no AI” contribution. Perhaps it’s because Homebrew has so many declarative guardrails and is so easy for agents to test.
I didn’t like the second part though where it starts banning people for “bad behavior”.
I get that AI resistance will be a thing in the next few years, but realistically engineers who plan on being around for the next 20-30 years just need to embrace it instead of being sour about it.
Because of machines, things have arguably improved for humanity as a whole. People who seek more elevated/finer products can still turn to handcrafted products.
Mass software will be produced by AI, and it will scale just fine.
I think you failed to notice the ban would be a reaction to recurrent bad behavior, such as ignoring maintainer requests.
What's your solution to handle accounts which spam projects with nonsense PRs?
> get that AI resistance will be a thing in the next few years (...)
This is a very lazy opinion to express. If you pay attention to the post you're replying to, you will notice that the problem isn't AI. The problem is spamming projects with PRs that are meaningless while completely ignoring maintainers.
You'd have the same problem if some guy decided to spam the project with a constant barrage of small low-effort nonfunctional PRs.
What will be happening is that bot will happily talk with your but, but falsely flagged people will be antagonized and eventually angry.
> you will notice that the problem isn't AI. The problem is spamming projects with PRs that are meaningless while completely ignoring maintainers.
What I noticed is that this is literally what AI companies encourage. This is intended use of AI.
You should really read the comments you are replying to. The whole point of the proposal is to provide actionable feedback to the people operating these drive-by PR bots, and kick out repeat offenders to prevent them from generating noise.
> What will be happening is that bot will happily talk with your but, but falsely flagged people will be antagonized and eventually angry.
I think you're trying too hard to come up with imaginary scenarios. Also, keep in mind that the objective is to steer contributions to be aligned with the project goals and help maintainers not waste time with low-quality noise. The goal is not to impose a fundamentalist militant stance against AI.
This plan will kick no bots, because bots dont mind discussion with another bot. However, falsely flagged people will realize they are talking with bot prompted to be obtuse, get annoyed anf then will be banned
> the objective is to steer contributions to be aligned with the project goals
That was not the objective as stated.
That AI will be given the power to ban people?
Or that AI will be the ones getting banned?
If it's the former, expect an invite from the AI resistance. :)
If it's the latter, well, that can be fixed by spending more tokens and writing better PRs so that the AI moderator doesn't think they're "low-effort, AI-like". Low-effort is low-effort, whether it's from humans or AI.
That's a very narrow view of the situation. I agree that this side of the coin exists, but there's also the other side where drinkable tap water is becoming a luxury in much of the global North, giant tornadoes and fires are sweeping the Earth, and temperatures are growing out of control.
So, to make lives more comfortable with machines for some millions of privileged people, they triggered the 6th mass extinction. That's the other side of the coin.
1. https://naturalhistory.si.edu/education/teaching-resources/p...
2. https://www.endangeredspeciesinternational.org/overview5.htm...
What's your thoughts on them banning bots?
Bots don't understand how their words can hurt people's feelings, and can even less understand that talking to a LLM is even more sad/infuriating for actual humans. I can handle CI/tests just fine, but LLMs playing on my human emotions will have me swearing in no time.
I contribute to free software projects because i want user control over the machine (that was the idea, long before LLMs). Slopware might technically be FLOSS, but it's certainly not in the spirit of FLOSS.
If I get a review at work from a bot that is annoying, misunderstands, etc I just ignore their reviews, or tell the bot to leave me alone. The bot is there to help me, and if it doesn't - that's not my problem. There's also always a human reviewer. For any automated feedback for PR's I think there should be "manual review" requests. If you disagree and the bot can decide that there is real human effort involved, then just let real humans also make the effort to review.
This is the key idea: to spend the human effort from maintainers, where there has been human effort from contributors. The AI should be able to determine the amount of human effort in a contribution. The bot should be an aid to make sure the human maintainer effort is spent only where there is human contributor effort.
That's a super weird way of putting it, which makes me hesitate to agree with you.
buuuut I agree that the LLM bot review experience is usually miserable, as the persona these things use is insufferable. It's a know-it-all only-speaks-in-imperatives thing, where the speaker lacks the actual merit that would grant them the right to that style.
LLMs can be very useful for code review, but the existing product implementations took a bit too many pages out of Kafka's works and are a bit too real in simulating a dysfunctional bureaucratic org.
Yes that's weird sorry. What i meant is it belongs in the uncanny valley. I don't have any good/bad feelings about test suites and CI feedback, it's very binary. Receiving a mail about a comment on my PR raises my hopes that it reached the maintainers (or interest from other contributors); receiving LLM text destroys my expectations.
Then, about the content itself, it has a very patronizing tone. Saying out loud every single thing that's in my +25/-3 PR is really infantilizing. I would probably have slightly less negative reactions if it only commented to suggest improvements, instead of patting me on the back. See this example: https://github.com/TwiN/gatus/pull/1712#pullrequestreview-46...
Since AI makes it easier for incompetent people to look smart, AI can help us expose those people for what they are.
I think if OSS maintainers start pushing that financial burden out to potential contributors then there will be a big outcry.
OSS may become pay-to-play.
Instead opensource is full of cynical projects and real important projects that do amazing work with maintainers who shoulder a great burden having to listen to people bitch about them for features and fixes while giving nothing and dealing with bad PR's (or even just having to evaluate good and mediocre PR's). Often the only pay is that you can go to HR and shit corp with your CV and say "Here is what I have done, can I now be treated like a bitch for more money".
I think opensource has allowed a lot businesses to treat developers with contempt and this is fully manifest with the way they conceive of AI agent's and Open source has probably become a vector for demoralization and devaluation more then anything.
Actually, if I see someone doing open source like it's a performative career checkbox, that will not be positive signal, and could easily be negative. I understand that people will do what they need to do to get a job, but that pragmatic career checkboxing itself isn't positive. I will have to look for positive signal elsewhere for that person.
The problem is that our industry has gotten very bad at hiring, and now we have everyone playing all sorts of games with it -- rehearsing Leetcode interviews, bad faith open source contributions, feigning enthusiasm, spamming AI-tweaked resumes, outright cheating on interviews -- rather than focusing on doing good work, and being part of a team.
If you're involved in hiring, and you care about effectiveness and culture, consider pushing back against the prevailing dysfunctional big-corporate-cattle-herding practices. Especially if your company has no excuse to be big-corporate dysfunctional, and can't afford to be.
Reminds me of this xkcd: https://www.explainxkcd.com/wiki/index.php/2899:_Goodhart%27...
After reading your comment, I had a thought that the key could be to separate random, drive-by open source contributions (which are easily gamed by pointing a bunch of AI agents at GitHub) from committing to maintain a small number of open source/community projects for the long term.
Helicopter parents and then young adults are already long-term gaming metrics, before they start high school, just to get into college, and it seems that the same kinds of thinking persist through first job and rest of career.
Being seen as a long-term maintainer of an open source project is now trivial, compared to some of the other things people already do.
Offhand, ideas:
1. Use criteria the metrics-gamers won't think of, and don't tell them what it is. (Why this might work: the metrics-gaming mindset might blind people to thinking outside that mindset. Why this might not work: you need someone not blinded to apply the criteria.)
2. Do things that are unattractive to the metrics-gamers, but attractive to people who aren't. (Though you might still get overwhelmed by near-zero-cost AI spamming applications anyway.)
3. Accept that too many current workers were brought up with metrics-gaming mindset, and the others are too hard to find in the noise, and select and align metrics-gamers in game-theoretic ways. (Extreme example: you pay minimum wage, and there are no promotions nor titles nor performance evaluation metrics not anything except success of the company, but everyone gets real equity that's worth zero unless the team together delivers and wins, in which case everyone gets equally wealthy. The trick would be convincing the dumber metrics-gamers that they can't just nod and game this, and if they don't play like a team member for the team to win, they will be voted off the island and get zero, and possibly left in a ditch.)