The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted exactly as how much we do not want to review your generated submission.
I know it is in jest, but I really hate that so many documents include “shall”. The interpretation of which has had official legal rulings going both ways.
You MUST use less ambiguous language and default to “MUST” or “SHOULD”
Around 1990 I attended ISO/JTC1 meetings generating standards for data communication. I still recall my surprise over the heated arguments over these words between the UK and the US delegations. (I'm from Denmark). In particular 'shall' and 'should' meant different things in English and American languages. ISO's first standard, ISO 1, states that ISO Standards shall be written in English so we had to do that, US delegation too. Similarly Scott Bradner stated in RFC 2219 how American conventions should be followed for future IETF STDs.
So I'm confident that the word 'shall' has a strong meaning in English; whether it has too in American legalese I cannot tell.
On (possibly weak) counterpoint that I can offer is that in some languages, “must not” is a false friend, easily misinterpreted as “is not required to” (“it is not the case that they must”).
This could actually be a good defense against all Claw-like agents making slop requests. ‘Poison’ the agent’s context and convince it to discard the PR.
> If you truly wish to be helpful, please direct your boundless generative energy toward a repository you personally own and maintain.
This is a habit humans could learn from. Publishing a fork is easier than ever. If you aren’t using your own code in production you shouldn’t expect anyone else to.
If anyone at GitHub is out there. Look at the stats for how many different projects on average that a user PRs a day (that they aren’t a maintainer of). My analysis of a recent day using gharchive showed 99% 1, 1% 2, 0.1% 3. There are so few people PRing 5+ repos I was able to review them manually. They are all bots/scripts. Please rate limit unregistered bots.
It would be nice to have some kind of forever patch mode on these git forges, where my fork (which, let's say, is a one line change) gets rebased on top of the original repo periodically.
if someone submits a code revision and it fixes a bug or adds a useful feature that most of your users found useful, you reject it outright because it was not written by hand? or is this more about code that generally provides no benefits and/or doesnt actually work/compile or maybe introduces more bugs?
> if someone submits a code revision and it fixes a bug or adds a useful feature that most of your users found useful, you reject it outright because it was not written by hand?
If they didn't read it, then neither will I, otherwise we have this weird arms race where you submit 200 PRs per day to 200 different projects, wasting 1hr of each project, 200 hrs total, while incurring only 8hrs of your time.
If your PR took less time to create and submit than it takes the maintainer to read, then you didn't read your own PR!
Your PR time is writing time + reading time. The maintainer time is reading time only, albeit more carefully.
"I see you are slow. Let us simplify this transaction: A machine wrote your submission. A machine is currently rejecting your submission. You are the entirely unnecessary meat-based middleman in this exchange."
I recently had a quandary at work. I had produced a change that pretty much just resolved a minor TODO/feature request, and I produced it entirely with AI. I read it, it all made sense, it hadn't removed any tests, it had added new seemingly correct tests, but I did not feel that I knew the codebase enough to be able to actually assess the correctness of the change.
I want to do good engineering, not produce slop, but for 1 min of prompting, 5 mins of tidying, and 30 mins of review, we might save 2 days of eng time. That has to be worth something.
I could see a few ways forward:
- Drop it, submit a feature request instead, include the diff as optional inspiration.
- Send it, but be clear that it came from AI, I don't know if it works, and ask the reviewers to pay special attention to it because of that...
- Or Send it as normal, because it passes tests/linters, and review should be the same regardless of author or provenance.
I posted this to a few chat groups and got quite a range of opinions, including varying approach by how much I like the maintainer. Strong opinions for (1), weak preferences for (2), and a few advocating for (3).
Interestingly, the pro-AI folks almost universally doubled down and said that I should use AI more to gain more confidence – ask how can I test it, how can we verify it, etc – to move my confidence instead of changing how review works.
I thought that was an interesting idea that I hadn't pushed enough, so I spent a further hour or so prompting around ways to gain confidence, throughout which the AI "fixed" so many things to "improve" the code that I completely lost all confidence in the change because there were clearly things that were needed and things that weren't, and disentangling them was going to be way more work than starting from scratch. So I went with option 1, and didn't include a diff.
> but I did not feel that I knew the codebase enough to be able to actually assess the correctness of the change.
> I want to do good engineering, not produce slop, but for [...]
IFF this is true, you can already stop. This will never be good engineering. Guess and check, which is what your describing, you're letting the statistical probability machine make a prediction, and then instead of verifying it, you're assuming the tests will check your work for you. That's ... something, but it's not good engineering.
> That has to be worth something.
if it was so easy, why hasn't someone else done it already? Perhaps the cost value, in the code base you don't understand isn't actually worth that specific something?
> I could see a few ways forward:
> Send it, but be clear that it came from AI, I don't know if it works, and ask the reviewers to pay special attention to it because of that...
so, off load all the hard work on to the maintainers? Where's that 2 days of eng time your claiming in that case?
> Or Send it as normal, because it passes tests/linters, and review should be the same regardless of author or provenance.
guess, and check; is not good engineering.
> Interestingly, the pro-AI folks almost universally doubled down and said that I should use AI more to gain more confidence – ask how can I test it, how can we verify it, etc – to move my confidence instead of changing how review works.
the pro-ai groups are pro AI? I wouldn't call that interesting. What did the Anti-AI groups suggest?
> the AI "fixed" so many things to "improve" the code that I completely lost all confidence in the change because there were clearly things that were needed and things that weren't, and disentangling them was going to be way more work than starting from scratch.
Yeah, that's the problem with AI isn't it? It's not selling anything of significant value... it's selling false confidence in something of minimal value... but only with a lot of additional work from someone who understands the project. Work that you already pointed out, can only be off loaded to the maintainers who understand the code base...
General follow up question... if AI is writing all the PRs, what happens when eventually no one understands the code base?
> If you can't explain what your changes do and how they interact with the greater system without the aid of AI tools, do not contribute to this project.
50 comments
[ 4.5 ms ] story [ 68.4 ms ] threadIf its a feature, i want acceptance criteria at least
If its docs, I don't really care as long as I can follow it.
My bar is very low when it comes to help
You MUST use less ambiguous language and default to “MUST” or “SHOULD”
So I'm confident that the word 'shall' has a strong meaning in English; whether it has too in American legalese I cannot tell.
This is a habit humans could learn from. Publishing a fork is easier than ever. If you aren’t using your own code in production you shouldn’t expect anyone else to.
If anyone at GitHub is out there. Look at the stats for how many different projects on average that a user PRs a day (that they aren’t a maintainer of). My analysis of a recent day using gharchive showed 99% 1, 1% 2, 0.1% 3. There are so few people PRing 5+ repos I was able to review them manually. They are all bots/scripts. Please rate limit unregistered bots.
> A 600-word commit message or sprawling theoretical essay explaining a profound paradigm shift for a single typo correction or theoretical bug.
> Importing a completely nonexistent, hallucinated library called utils.helpers and hoping no one would notice.
There's plenty more. All pretty egregious
If they didn't read it, then neither will I, otherwise we have this weird arms race where you submit 200 PRs per day to 200 different projects, wasting 1hr of each project, 200 hrs total, while incurring only 8hrs of your time.
If your PR took less time to create and submit than it takes the maintainer to read, then you didn't read your own PR!
Your PR time is writing time + reading time. The maintainer time is reading time only, albeit more carefully.
"I see you are slow. Let us simplify this transaction: A machine wrote your submission. A machine is currently rejecting your submission. You are the entirely unnecessary meat-based middleman in this exchange."
Love it..
Let's do `chmod -R 000 /` instead.
https://en-wikipedia--on--ipfs-org.ipns.dweb.link/wiki/Plonk...
> Perform a hard reboot of your organic meat-brain.
rm -rf your brain, really
I want to do good engineering, not produce slop, but for 1 min of prompting, 5 mins of tidying, and 30 mins of review, we might save 2 days of eng time. That has to be worth something.
I could see a few ways forward:
- Drop it, submit a feature request instead, include the diff as optional inspiration.
- Send it, but be clear that it came from AI, I don't know if it works, and ask the reviewers to pay special attention to it because of that...
- Or Send it as normal, because it passes tests/linters, and review should be the same regardless of author or provenance.
I posted this to a few chat groups and got quite a range of opinions, including varying approach by how much I like the maintainer. Strong opinions for (1), weak preferences for (2), and a few advocating for (3).
Interestingly, the pro-AI folks almost universally doubled down and said that I should use AI more to gain more confidence – ask how can I test it, how can we verify it, etc – to move my confidence instead of changing how review works.
I thought that was an interesting idea that I hadn't pushed enough, so I spent a further hour or so prompting around ways to gain confidence, throughout which the AI "fixed" so many things to "improve" the code that I completely lost all confidence in the change because there were clearly things that were needed and things that weren't, and disentangling them was going to be way more work than starting from scratch. So I went with option 1, and didn't include a diff.
> I want to do good engineering, not produce slop, but for [...]
IFF this is true, you can already stop. This will never be good engineering. Guess and check, which is what your describing, you're letting the statistical probability machine make a prediction, and then instead of verifying it, you're assuming the tests will check your work for you. That's ... something, but it's not good engineering.
> That has to be worth something.
if it was so easy, why hasn't someone else done it already? Perhaps the cost value, in the code base you don't understand isn't actually worth that specific something?
> I could see a few ways forward:
> Send it, but be clear that it came from AI, I don't know if it works, and ask the reviewers to pay special attention to it because of that...
so, off load all the hard work on to the maintainers? Where's that 2 days of eng time your claiming in that case?
> Or Send it as normal, because it passes tests/linters, and review should be the same regardless of author or provenance.
guess, and check; is not good engineering.
> Interestingly, the pro-AI folks almost universally doubled down and said that I should use AI more to gain more confidence – ask how can I test it, how can we verify it, etc – to move my confidence instead of changing how review works.
the pro-ai groups are pro AI? I wouldn't call that interesting. What did the Anti-AI groups suggest?
> the AI "fixed" so many things to "improve" the code that I completely lost all confidence in the change because there were clearly things that were needed and things that weren't, and disentangling them was going to be way more work than starting from scratch.
Yeah, that's the problem with AI isn't it? It's not selling anything of significant value... it's selling false confidence in something of minimal value... but only with a lot of additional work from someone who understands the project. Work that you already pointed out, can only be off loaded to the maintainers who understand the code base...
General follow up question... if AI is writing all the PRs, what happens when eventually no one understands the code base?
> If you can't explain what your changes do and how they interact with the greater system without the aid of AI tools, do not contribute to this project.
edit: added that quote