I think this is actually a bad idea, especially the language and tone part.
You can not detect AI writing by the language and tone, all LLMs are trained and prompted to write in a very particular style. You can just tell them to write in a different style and they will. What is worse that the default LLM writing style is actually quite common. If you read through that list you will also see that many of these are very much human errors.
Trying to detect what is and isn't LLM generated text will only lead to people chasing ghosts, either accusing innocent people or putting faith in text which is the result of more careful prompting.
This is purely anecdotal, but I think I’ve seen ChatGPT insert special space characters other than normal space. It also likes to use the different dash characters (en, em and hyphen) more than would appear in normal text.
There are two major problems with Wikipedia doing this:
1. False Positives: phrases like "on the other hand", "not only x but y" are definitely used by humans. You can't simply accuse others for using AI by just checking some phrases to be in text. I mean AI itself is trained on text written by humans, so the reason it uses those phrases is because they are more common in it's training set.
2. By making a set of what seems like AI, they give people the opportunity to just tell AI what phrases NOT to use. Every person who prompts to AI, can use it to make it more like human. Ironically, what the wikipedia itself was trying to stop.
I think a lot of people are missing a crucial point here; the main problem with llm's (as far as wiki is concerned) is these ways of writing are biased, weasel wordy, puffery etc etc. which wiki doesn't want to have regardless of who wrote it.
Technically speaking, if an llm can write wp style prose and source it correctly, that wouldn't be a problem (imo)
I sniff that guidelines like this are going to disenfranchise the language of marketing copy and other consumer-orientated lingo.
The advertisement wave of the future will be similar to when Nike and Virgil Abloh were putting out sneakers that said "SHOE" on them. Or something like that.
8 comments
[ 0.26 ms ] story [ 26.7 ms ] threadYou can not detect AI writing by the language and tone, all LLMs are trained and prompted to write in a very particular style. You can just tell them to write in a different style and they will. What is worse that the default LLM writing style is actually quite common. If you read through that list you will also see that many of these are very much human errors.
Trying to detect what is and isn't LLM generated text will only lead to people chasing ghosts, either accusing innocent people or putting faith in text which is the result of more careful prompting.
1. False Positives: phrases like "on the other hand", "not only x but y" are definitely used by humans. You can't simply accuse others for using AI by just checking some phrases to be in text. I mean AI itself is trained on text written by humans, so the reason it uses those phrases is because they are more common in it's training set.
2. By making a set of what seems like AI, they give people the opportunity to just tell AI what phrases NOT to use. Every person who prompts to AI, can use it to make it more like human. Ironically, what the wikipedia itself was trying to stop.
Technically speaking, if an llm can write wp style prose and source it correctly, that wouldn't be a problem (imo)
The advertisement wave of the future will be similar to when Nike and Virgil Abloh were putting out sneakers that said "SHOE" on them. Or something like that.
The working title of this trend is "Bruxism".