It's in-line with the 'nanny' stereotype of the Rust community that they give you permission to act in a way they would never be able to verify anyways:
> The following are allowed.
> Asking an LLM questions about an existing codebase.
> Asking an LLM to summarize comments on an issue, PR, or RFC...
Like seriously, what's the point of explicitly allowing this? Imagine the opposite were true, you weren't allowed to do this - what would they do? Revert an update because the person later claimed they checked it with an LLM?
The Linux policy on this is much superior and more sensible.
This is highly interesting. It seems clear to me that a lot of thought and work went into this. If I ever were to write a similar document, I'm sure I could learn a lot from this one. Props to the authors and all involved.
Note that there are currently several proposed policies (plus hundreds of discussions mostly in private channels), and frankly I'm not sure we'll ever reach a consensus (I'm a Rust project member).
This policy is straightforward and shouldn't be particularly controversial (I'm sure it will be bikeshedded to death though). It basically bans the obvious stuff ("don't just drop LLM generated comments onto PRs") and allows the important stuff like LLMs writing code so long as you disclose.
edit: Wow people did not read the policy. It's literally just "if you use an LLM you are responsible for it, we will reject low quality PRs, please disclose that you have used an LLM". This is bog standard.
> Using an LLM to discover bugs, as long as you personally verify the bug, write it up yourself, and disclose that an LLM was used.
What are they going to do go back and reject a bug if someone later admits they found it with an LLM? Honestly they and most other project would probably be better off just ignoring the situation until norms start developing.
Kudos to the team for this. I think it’s brave of them to stand up for their own experiences and push back against the hype train.
Before you knee jerk hate on the team for being luddites, consider:
1. For a language like rust there’s too few eyes and too many mouths. Reviewing is a job, and is extremely taxing.
2. The code base needs to be highly hermetic because it’s load bearing across the global economy
3. Most changes are only relevant if they’ve followed extensive process, including community feedback.
I'm not gonna say no to some system that validates my code before I present it as a PR, and if said system (it being static checking, dynamic checking, or LLM), gives me a 'comment', I will interpret the output, validate, and decide whether to take action, and how.
And maybe there I disagree a little with the proposal. It's me, it's my code. I stand behind it. But I get where the authors come from, and I believe it's a fine compromise.
I think more importantly in the world of Software Engineering we're seeing a split.
On one hand, people who go all-in on AI and take the output as 100% correct, copy-paste it as theirs without reviewing, prompt it to create PRs and submit them as-is, and worse, as their own.
Then there is the other side, people who use AI as a tool to validate and go deeper.
The problem with the first group that everywhere they feel entitled to shift the validation onus from themselves to the receipient of what they're sending, it being PR or review comment or message or whatever.
In PRs now the repo maintainer has to do a lot more work, as they cannot rely on the social construct of "OptionOfT wrote this, they have 10+ years of experience in these and these systems, so we can look at the PR through that lens".
Equally, I've been on the receiving end of AI PR comments (the PR was human-authored), but copy-pasted by humans, presenting the comment as their own, without properly (if anything) validating the correctness of the comment, or whether it actually makes sense for the PR. Lots of derailment there. This now increases the workload of the PR author, as above. We need to validate and cannot rely on the social construct. Is the comment even correct? What's the context? Why? Is it a hallucination?
And the downside is that it looks like the first group is now going faster, but the second group is actually slowing down due to the increased burden.
20 comments
[ 2.4 ms ] story [ 41.6 ms ] thread> These are organized along a spectrum of AI friendliness, where top is least friendly, and bottom is most friendly.
This section is an extremely useful reference
https://github.com/jyn514/rust-forge/blob/llm-policy/src/pol...
It's in-line with the 'nanny' stereotype of the Rust community that they give you permission to act in a way they would never be able to verify anyways:
> The following are allowed. > Asking an LLM questions about an existing codebase. > Asking an LLM to summarize comments on an issue, PR, or RFC...
Like seriously, what's the point of explicitly allowing this? Imagine the opposite were true, you weren't allowed to do this - what would they do? Revert an update because the person later claimed they checked it with an LLM?
The Linux policy on this is much superior and more sensible.
Will it fix a related but different problem? Likely.
> People must be vouched for before interacting with certain parts of a project (the exact parts are configurable to the project to enforce).
https://github.com/mitchellh/vouch
I think many projects will adopt this instead of allowing everyone / blocking everyone
Many projects have "ai slop" check in place to directly close and ban user if it is "ai slop". Else, it will be hard to handle the velocity of PRs
edit: Wow people did not read the policy. It's literally just "if you use an LLM you are responsible for it, we will reject low quality PRs, please disclose that you have used an LLM". This is bog standard.
> Using an LLM to discover bugs, as long as you personally verify the bug, write it up yourself, and disclose that an LLM was used.
What are they going to do go back and reject a bug if someone later admits they found it with an LLM? Honestly they and most other project would probably be better off just ignoring the situation until norms start developing.
Before you knee jerk hate on the team for being luddites, consider:
1. For a language like rust there’s too few eyes and too many mouths. Reviewing is a job, and is extremely taxing. 2. The code base needs to be highly hermetic because it’s load bearing across the global economy 3. Most changes are only relevant if they’ve followed extensive process, including community feedback.
I'm not gonna say no to some system that validates my code before I present it as a PR, and if said system (it being static checking, dynamic checking, or LLM), gives me a 'comment', I will interpret the output, validate, and decide whether to take action, and how.
And maybe there I disagree a little with the proposal. It's me, it's my code. I stand behind it. But I get where the authors come from, and I believe it's a fine compromise.
I think more importantly in the world of Software Engineering we're seeing a split.
On one hand, people who go all-in on AI and take the output as 100% correct, copy-paste it as theirs without reviewing, prompt it to create PRs and submit them as-is, and worse, as their own.
Then there is the other side, people who use AI as a tool to validate and go deeper.
The problem with the first group that everywhere they feel entitled to shift the validation onus from themselves to the receipient of what they're sending, it being PR or review comment or message or whatever.
In PRs now the repo maintainer has to do a lot more work, as they cannot rely on the social construct of "OptionOfT wrote this, they have 10+ years of experience in these and these systems, so we can look at the PR through that lens".
Equally, I've been on the receiving end of AI PR comments (the PR was human-authored), but copy-pasted by humans, presenting the comment as their own, without properly (if anything) validating the correctness of the comment, or whether it actually makes sense for the PR. Lots of derailment there. This now increases the workload of the PR author, as above. We need to validate and cannot rely on the social construct. Is the comment even correct? What's the context? Why? Is it a hallucination?
And the downside is that it looks like the first group is now going faster, but the second group is actually slowing down due to the increased burden.