> Betteridge's law of headlines is an adage that states: "Any headline that ends in a question mark can be answered by the word no." It is named after Ian Betteridge, a British technology journalist who wrote about it in 2009, although the principle is much older. It is based on the assumption that if the publishers were confident that the answer was yes, they would have presented it as an assertion; by presenting it as a question, they are not accountable for whether it is correct or not.
For the last few days I have been wondering if there is any analysis to see if this "law" is accurate.
Is there anything other than this?
> With 46% non-polar and 20% answered “yes”, at least two thirds of our headline sample violates Betteridge’s law. We conclude that it cannot be “mostly correct” either.
Well, to me it sounds vague rather than fair. "Success" in what? In predicting new observations? Or in securing research grants? Or in letting people put the word "astonishing" in papers?
It just muddles things up to use qualitative terms in scholarly articles like that and it's standard advice to graduates to avoid it. And those who don't follow the advice have it repeated to them by reviewers. And that seems to be a good thing. Personally, I don't want to be told what is "astonishing", and, consequently, what isn't. I am perfectly capable of being astonished, or not, all on my own.
See, it's the "show, don't tell" principle. Astonish people, but don't tell them they're astonished, or they very likely won't.
I don’t think text is sufficiently unconstrained. It is trivial to get GPT to output “Hi, how are you?” but of course it’s impossible to determine if that common turn of phrase is human or machine generated.
There is too much a correlation between the position of tokens and correct meaning. In contrast, in an image, it’s very possible to have pixels shuffle around and not change the look. There, it is more likely that we can determine that the output is statistically more likely to be from a particular model.
Yeah, trivial text samples are obviously insufficient. At the other extreme (say millions of tokens of output), there's probably a reasonable chance that you could reliably distinguish human and LLM output.
I suspect there might be some interesting questions down the rabbit hole in the middle though.
Doesn't it boil down to a halting-problem kind of argument?
Assume you have a model that can detect AI-generated text. Then you can build a model that emulates the detection model, finds the nearest non-detected text, and outputs it.
Only if your detection system is "perfect". Personally I'd be fine with something with high precision and low recall, even if it means forcing users to rewrite their content to make it less AI-like (like /r9k/ on steroids).
Funny you mention this because more threads/comments on Reddit are bot created. It's unfortunate that the thing that killed forums is now being filled with bots replying to each other and posting threads.
It's alright for now, but not so sure how long that will remain. I already stumbled upon some of them in a niche subreddit. Easy to detect now, due to the uncanny way it interjects "facts" into it's comments that are supposed to mimic a reddit rant. In the long term when this stops being so easy to detect, I think a lot of people will start questioning why use Reddit (or similar places) at all.
I think also if you hate SEO spam, you might want to brace yourself for chatGPT enabled SEO spam. It's so "good" you might think "Dr. Brandon Amersmith" is a real person and take horrible advice from him.
Yeah I mean if AI HN comments start to appear does that really matter in the end. Ie there’s already a lot of noise in the form of low quality comments, and they get downvoted. If an AI manages to produce high quality comments that people upvote im not sure I care that it wasn’t generated by a human
It would matter to me because I'm here to read what people write, not what bots write. Besides that being against the HN guidelines noise in the form of high quality comments is still noise.
"High quality" in HN terms means it's not just a couple of words or a meme, isn't glaringly false or advertising or particularly off topic or antagonistic.
"Feedback" that isn't actually feedback, syntheses of other people's comments that claim to be a new person adding their agreement, anecdotes that didn't happen, little known facts that aren't true and pseudo-recommendations that are actually derived largely from the product marketing copy will usually pass this filter, but they're definitely noise
I'm here to read high quality comments. If I'm unable to distinguish between high and low quality comments then I have another problem... maybe I'm the one that's not human.
I think what he meant was that an AI generated comment might be high quality bullshit, like for example a story from personal experience that has never existed, or a false fact that looks convincing and/or is hard to check, or a personal opinion written by a statistical meat grinder, etc. And these comments might come to dominate any discussion because they are so easy to generate, unlike human high quality comments.
If it's now easy to generate nice sounding bullshit then perhaps we should be re-evaluating our goals in reading that. The examples you give is no different from going on TikTok and ingesting mindless entertaining content.
Exactly. We don’t want HN turning into TikTok, do we? What’s worse, there will inevitably appear AI constructed “native” ads or even political propaganda - high quality comments skillfully promoting products or someone’s agenda.
Yes, so downvote comments that are just entertainment. Only keep high quality objective comments. At best problem solved, at worst AI are better at commenting than humans.
It does matter. For instance, what will you train your future AI on? If AI ends up eating its own tail it will end up amplifying its errors and the end result is that you won't be able to trust anything that wasn't written down before this hit.
You shouldn't be trusting random comments as truth in the first place. Maybe learn to identify high value first? Then you can just train on high value targets. Just because (hypothetically) 4-chan is full of humans doesn't make it suddenly a good training target for AI.
Random comments attached to accounts with good reputation count for something. If those get replaced by random comments written by bots attached to accounts with good reputation then that would be a problem. High value has many signals, reputation is one of them.
4chan is/was a very mixed bag, and just like there are garbage HN comments so there were excellent 4chan comments.
OpenAI training data oversamples sites which got 3 reddit likes[1], because it improves model. Most large scale training data has some filter on commoncrawl.
My hope is that future AI will become smarter, not just trained on more data, and as a result it will be capable to perhaps “rate” the quality of any text, based on fact checking, reputation of the commenter, content of all their other comments, frequency of the previous comments, consistency of the style, availability of particular LLMs at the time, etc. It won’t be good enough to detect AI of the same level, but might be good enough to filter out most of the earlier AI’s comments. GPT-5 might be able to detect comments generated by GPT-3, even if its training dataset included a small fraction of GPT-3 generated content, or even if it was trained on the same corpus as GPT-4 (up to 2021).
Why is "at schools" necessarily important though? Wouldn't it also be nice to know if the scientific articles you are reading, or the code a company is based upon, or a new piece of legislature was constructed from someone just asking an LLM to do it for them?
As usual, "just" pulls a lot of weight - if they're putting the straight LLM output, it probably won't be great regardless.
If they put effort into making the ouput good, then chances are the content will be good regardless of whether they wrote it by hand, or wrote it using an LLM.
Like, I don't need to know whether you used a bic ballpoint pen, or some other ballpoint pen to write your note on paper; the tool is a tool
Plagiarism can be solved with non-AI methods because plagiarism is defined more or less by exact copying. If you mean detection of AI generated essays at schools, then teachers just have to expect better essays and be able to identify them since students now have a new tool.
Think of it like grading math homework before and after the invention of calculators. Once students had calculators at home the problems simply got harder and higher level, since there was no reasonable way to enforce "don't use a calculator at home with these artificially simplistic problems."
The more I think about this, the less I agree with it.
Think of all the online communication you've had. The good parts. The great but messy blog post that changed your mind about something. Communicating with someone on a social network only to share each other's work and establish a great working relationship. Talking to a friend in DMs sharing your pain. Getting feedback from a mentor. Having people tell you how amazing your art is or how it inspired them. The bad parts too. The negative comments from random people. Others mocking your tweet. People calling your HN comment idiotic.
All of these things affect us because we assume a person is behind them, and therefore their intent is clear. Without that, we devalue all communication to compensate. Like a more exaggerated effect of how we view likes on social media, where "20k likes" means nothing.
Truly believe we're only ok with it now because it's easy to tell what is AI generated, and they're mostly confined. When AI is creating videos on Twitter, creating threads on Reddit, posting blogs on their personal site to be shared to Mastodon, replying to other bots on HN... I'm not sure we'll feel that it's whether we can detect quality anymore.
There's also the differentiation theory where in order to separate human connection from bots, we'll do the opposite. We'll start incorporating prompt injections into everything we post, start purposefully being "low quality" because bots are trained against that. Now we're creating a very particular kind of societal mess.
I definitely don't agree with the GP but I don't agree with you either. Human could and absolutely become attached to non humans, ranging from dog to pen to house. And I believe humans might begin to develop deeper relationship with AI bots very soon.
I don't know. Part of why we anthropomorphize so much is because we can still feel the physical elements. We pet a dog, we live inside our homes, list goes on. When that is not possible, how does this change people and their valuation of online communication? Would you still love "Anne" when she's now rejected your 50th attempt to meet in person? Would you still reply to comments like mine if the default assumption wasn't "that's another human being I can persuade" and instead "that is a bot"?
I'm sure in the long term that differentiation alone would change, you know androids and all that jazz - however in the meantime, we're going to be facing a tremendous shift in how we interact online.
Dogs are very precious things. As is a house. However, anytime we draw a line in the sand and say “this is the line that separates humanity from computers” we always seem to cross it.
I really think our efforts as a species would be really better off going towards climate change mitigation and adaption, not burning the world up faster by running thousands of not millions of GPUs on AI experiments and moonshots.
What you will instead find is the development of ML assisted stylometry. Not to detect ML but to detect when the writing style is not in line with how some person or persons (known employees or students for example) normally write. Or you might be required to provide access to your social media to compare how you normally post stuff.
One under-discussed component of Snowden's XKEYSCORE leak was the claim made in one of the presentations that the NSA has enough stylometry data collected that, give some piece of text, they can not only tell you who wrote it, but they can also tell you where they wrote it (in 2008).
Inferred from the "Document Tracking" slide about 2/3rds of the way in.
It will be interesting to see if any research projects start seriously looking into styleometry using LLMs, but that's basically what you'd need to do to actually generate something useful detection-wise. Ideally it would light up a document identifying the style of known humans, as well as the style of various LLMs. If teachers actually start cracking down on this sort of thing, I'm sure people will fine tune models on their own text, ensuring that anything written uses their personal style, but we can tackle that once we get there.
You could take a similar approach to detection (hypothetically). You provide writing samples a LLM is fine tuned on, then future texts could possibly be filtered by perplexity or similar to detect obvious differences in style. Of course, there would be side effects, and ideally we don’t start creating incentives where we don’t need to. Man, I’m glad I learned to write before all this.
Just a guess, but it would not be surprising if there are variations in stylistic writing that are influenced by environment (location, people interacted with, air quality, etc). I have certainly observed it happen with myself based on who I've interacted with recently.
My assumption was that it would first detect when it was written using stylometry, and the change in style over time, and could use that, in addition to other location tracking data in XKEYSCORE to figure out where the target was at that time. That might be giving them a bit too much credit. They could just be using some other simple heuristics to guess at location data. Hard to tell.
XKEYSCORE is/was basically the NSA's Google, and in theory contains nearly all of the data ever collected from the various collection points all over the internet.
Unless I am misunderstanding the document, they aren't claiming to be able to read a document and tell you who wrote it. They claim they have a massive database of when documents have been shared and then they go back to find the original source that way. Like chainalysis for evil people.
How will websites and third party intermediaries, e.g., so-called "tech" companies, distinguish users submitting human-generated input from users submitting AI-generated input.
First party platform like Google, Facebook or office 365 easily could with high accuracy by measuring typing rate of user, recording speech etc. At some point, data in the internet might not be trusted at all and the data about the source of the text could be worth billions.
Typing rate could be controlled using something like a USB Rubber Ducky. A user could emulate someone else's typing rate. As for speech, that could be synthesised with something like VALL-E.
Detecting AI-generated text is an adversarial process. Meaning there are going to exist people interested in avoiding detection. This just ensures there is never going to be a reliable solution. There will be some solutions but none that will be reliable.
We still have counterfeit banknotes and email spam. Seems like super easy problem to fix but there are always some banknotes and always some email spam that avoid detection. And this is simply because whenever somebody finds a way to close the current hole, somebody else finds a way to open another.
There is targeted spam now in my own language and location; it gets past the filters because it doesn't look like spam at all. I think most spam gets flagged as spam, not because the text context but because of the from/to server / domain / email address.
Ever since December, I've noticed a massive uptick in spam getting past my Gmail filter. We are talking 100+ emails per day selling sexual enhancement drugs, marital aids, conspiranoid prepper nonsense, money mule scams, crypto extortion and various other drivel.
What they all have in common is that they're seemingly going through some kind of spinning algorithm to word them differently enough to avoid the filter picking them up.
Agreed. And it's also the wrong way to approach this problem. We don't need to stop AI text, if some kid on reddit thinks it's funny to run an account posting GPT-3 replies that by itself isn't an issue. Frankly some of the human interactions aren't any more useful than what you'd get with a bot, on the contrary even in some cases.
The big danger is the internet getting flooded with machine generated content. All social networks, all of Wikipedia, everything. To stop this we need sybil resistant sign-up processes. We need to verify the user running the account is an individual human who does not control 1000 other accounts through automation. There needs to be proof of personhood and we should have had this yesterday already. Sites like Twitter and Facebook are swarming with bot accounts as we speak.
There are two futures: One is Sam Altman taking everyone's iris scans (I'm not making this up). In the other we have privacy preserving decentralized solutions to proving our unique humanity.
On flooding the Internet with generated content, the cat is out of the bag. There are already people paid to create lots of "personal" accounts and then create content like posts or reviews. This entire "industry" is getting transformed as we speak to AI-generated content.
I think this is unavoidable.
I think (at least I really wish) there is a chance that young people will rebel and will decline using the Internet and instead learn to cherish personal contact. It make take a generation for this to happen (ie. current children to find out how miserable the Internet is in couple of years). It is also possible that certified human-generated content will become a luxury that you will have to pay extra.
It may take some time for people to catch on, but they will find that trying to interact with a website flooded with bots is meaningless, rewardless. Even Internet trolls will stop getting their kick. Internet will become a cesspool where nobody ventures for fun because there is no fun to be had.
It's not the judgement about interests around a type of process, but the fundamental process itself: it leaves no "tooling marks." There is no way to see it was "made by a human" or "made by an algorithm" because there is no metadata and no telltale marks left in the pure data of a string of text.
Perhaps in a fit of reactionary over-regulation overreach and wishful half-measures, governments decide to make it a very serious crime to generate anything by AI without submitting a signature to a planetary repository of AI generated works. It won't work for obvious impractical, unenforceable, and spread of variations but it will make them feel like they've "tamed" a disruption when, like climate change, they've done absolutely nothing.
Nope. "Aaaaaaaaaaaaaaaaaaaaaaaa". Is this an AI generated text? ChatGPT is a glorified form letter, and there is no way to detect form letters if you only have one sample. I remember broken form letter e-mails starting with "Dear null" and similarly broken AI responses can be detected. If ChatGPT model never changed, one can eventually infer certain signatures in its responses. But of course it will be technically improved and trained with more data over time. How can one detect an arbitrary moving target?
The article argues that the current detection methods for AI-generated text are unreliable, which poses a significant risk for the malicious use of LLMs. This is a controversial topic, as some may argue that the benefits of LLMs outweigh the potential risks. However, as the technology continues to advance, we need to have an honest conversation about the ethical use of AI-generated text and the potential consequences of unregulated use. Ultimately, it's up to society to decide whether the risks are worth taking.
I was thinking the same thing. If you ask ChatGPT something without asking for a change in style of response it seems to always spit out a response like this.
A format where there’s an intro, middle sentence, middle sentence that is about the opposing view, concluding sentence. Written in a slightly persuasive non-combatant way.
Can't wait for this to bring about a fresh hell of emergent human behavior.
Hmm. My posts on social media keep getting weighted down because it seems too much like ChatGPT. That means (laughing) I need to start writing in a very strange (weird) style so that I throw off the AI-detection algorithms.
Or gosh, students writing papers, and the false positives there... "Sorry kid, Turn-It-In says your paper was written by an AI, try again."
"In this paper, we show both theoretically and empirically, that these state-of-the-art detectors cannot reliably detect LLM outputs in practical scenarios."
This was my finding as well. I abandoned this effort when I found that any created model would be inaccurate on small texts and could be circumvented with a bit of prompt engineering.
The only way it's possible is if the LLM that generated the text is proprietary and the company behind it provides a way to see if a given text sample was ever generated by said LLM. Otherwise, no. It's definitionally impossible.
98 comments
[ 3.0 ms ] story [ 157 ms ] threadWhich is what the paper reduxes to.
> Betteridge's law of headlines is an adage that states: "Any headline that ends in a question mark can be answered by the word no." It is named after Ian Betteridge, a British technology journalist who wrote about it in 2009, although the principle is much older. It is based on the assumption that if the publishers were confident that the answer was yes, they would have presented it as an assertion; by presenting it as a question, they are not accountable for whether it is correct or not.
https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
Is there anything other than this?
> With 46% non-polar and 20% answered “yes”, at least two thirds of our headline sample violates Betteridge’s law. We conclude that it cannot be “mostly correct” either.
http://calmerthanyouare.org/2015/03/19/betteridges-law.html#....
Or you can just apply betteridge's law to it. ;-)
It just muddles things up to use qualitative terms in scholarly articles like that and it's standard advice to graduates to avoid it. And those who don't follow the advice have it repeated to them by reviewers. And that seems to be a good thing. Personally, I don't want to be told what is "astonishing", and, consequently, what isn't. I am perfectly capable of being astonished, or not, all on my own.
See, it's the "show, don't tell" principle. Astonish people, but don't tell them they're astonished, or they very likely won't.
Especially with GPT4 and the right prompt.
There is too much a correlation between the position of tokens and correct meaning. In contrast, in an image, it’s very possible to have pixels shuffle around and not change the look. There, it is more likely that we can determine that the output is statistically more likely to be from a particular model.
I suspect there might be some interesting questions down the rabbit hole in the middle though.
[0] https://arxiv.org/abs/2303.08774
Doesn't it boil down to a halting-problem kind of argument?
Assume you have a model that can detect AI-generated text. Then you can build a model that emulates the detection model, finds the nearest non-detected text, and outputs it.
Sometimes I could see why if I went back and pixel peeped the training data, but sometimes I couldn't.
I wonder if something similar can be applied to LLMs? Maybe running its own output through the net would "amplify" it in a detectable way.
It's alright for now, but not so sure how long that will remain. I already stumbled upon some of them in a niche subreddit. Easy to detect now, due to the uncanny way it interjects "facts" into it's comments that are supposed to mimic a reddit rant. In the long term when this stops being so easy to detect, I think a lot of people will start questioning why use Reddit (or similar places) at all.
I think also if you hate SEO spam, you might want to brace yourself for chatGPT enabled SEO spam. It's so "good" you might think "Dr. Brandon Amersmith" is a real person and take horrible advice from him.
"Feedback" that isn't actually feedback, syntheses of other people's comments that claim to be a new person adding their agreement, anecdotes that didn't happen, little known facts that aren't true and pseudo-recommendations that are actually derived largely from the product marketing copy will usually pass this filter, but they're definitely noise
4chan is/was a very mixed bag, and just like there are garbage HN comments so there were excellent 4chan comments.
[1]: https://paperswithcode.com/dataset/webtext
If they put effort into making the ouput good, then chances are the content will be good regardless of whether they wrote it by hand, or wrote it using an LLM.
Like, I don't need to know whether you used a bic ballpoint pen, or some other ballpoint pen to write your note on paper; the tool is a tool
Think of it like grading math homework before and after the invention of calculators. Once students had calculators at home the problems simply got harder and higher level, since there was no reasonable way to enforce "don't use a calculator at home with these artificially simplistic problems."
Think of all the online communication you've had. The good parts. The great but messy blog post that changed your mind about something. Communicating with someone on a social network only to share each other's work and establish a great working relationship. Talking to a friend in DMs sharing your pain. Getting feedback from a mentor. Having people tell you how amazing your art is or how it inspired them. The bad parts too. The negative comments from random people. Others mocking your tweet. People calling your HN comment idiotic.
All of these things affect us because we assume a person is behind them, and therefore their intent is clear. Without that, we devalue all communication to compensate. Like a more exaggerated effect of how we view likes on social media, where "20k likes" means nothing.
Truly believe we're only ok with it now because it's easy to tell what is AI generated, and they're mostly confined. When AI is creating videos on Twitter, creating threads on Reddit, posting blogs on their personal site to be shared to Mastodon, replying to other bots on HN... I'm not sure we'll feel that it's whether we can detect quality anymore.
There's also the differentiation theory where in order to separate human connection from bots, we'll do the opposite. We'll start incorporating prompt injections into everything we post, start purposefully being "low quality" because bots are trained against that. Now we're creating a very particular kind of societal mess.
I'm sure in the long term that differentiation alone would change, you know androids and all that jazz - however in the meantime, we're going to be facing a tremendous shift in how we interact online.
We need “AI” because we’re collectively stupid.
1.) the ability to MATHS in our heads.
2.) Navigation biologically (internal compass) - and map reading ability.
3.) analog communication methods
4.) FOOD (commerce, growing, etc)
-
even if we are "'This is Fine'" -- The above is still valuable.
Inferred from the "Document Tracking" slide about 2/3rds of the way in.
https://www.theguardian.com/world/interactive/2013/jul/31/ns...
It will be interesting to see if any research projects start seriously looking into styleometry using LLMs, but that's basically what you'd need to do to actually generate something useful detection-wise. Ideally it would light up a document identifying the style of known humans, as well as the style of various LLMs. If teachers actually start cracking down on this sort of thing, I'm sure people will fine tune models on their own text, ensuring that anything written uses their personal style, but we can tackle that once we get there.
Joking.
XKEYSCORE is/was basically the NSA's Google, and in theory contains nearly all of the data ever collected from the various collection points all over the internet.
Simple option is that someone could even handtype GPT content, but less than 0.1% or so will do that.
Detecting AI-generated text is an adversarial process. Meaning there are going to exist people interested in avoiding detection. This just ensures there is never going to be a reliable solution. There will be some solutions but none that will be reliable.
We still have counterfeit banknotes and email spam. Seems like super easy problem to fix but there are always some banknotes and always some email spam that avoid detection. And this is simply because whenever somebody finds a way to close the current hole, somebody else finds a way to open another.
Even after a decade, it's really, really rare email spam to get past the gmail filter (at least in my experience)
I mark everything that manages to bypass it as spam, but the day after almost identical spam messages manage to break through anyway...
Usually it's because it's of popular spam subjects, spoofed domains, untrustworthy email servers, etc.
What they all have in common is that they're seemingly going through some kind of spinning algorithm to word them differently enough to avoid the filter picking them up.
The big danger is the internet getting flooded with machine generated content. All social networks, all of Wikipedia, everything. To stop this we need sybil resistant sign-up processes. We need to verify the user running the account is an individual human who does not control 1000 other accounts through automation. There needs to be proof of personhood and we should have had this yesterday already. Sites like Twitter and Facebook are swarming with bot accounts as we speak.
I think this is unavoidable.
I think (at least I really wish) there is a chance that young people will rebel and will decline using the Internet and instead learn to cherish personal contact. It make take a generation for this to happen (ie. current children to find out how miserable the Internet is in couple of years). It is also possible that certified human-generated content will become a luxury that you will have to pay extra.
It may take some time for people to catch on, but they will find that trying to interact with a website flooded with bots is meaningless, rewardless. Even Internet trolls will stop getting their kick. Internet will become a cesspool where nobody ventures for fun because there is no fun to be had.
https://worldcoin.org/
Perhaps in a fit of reactionary over-regulation overreach and wishful half-measures, governments decide to make it a very serious crime to generate anything by AI without submitting a signature to a planetary repository of AI generated works. It won't work for obvious impractical, unenforceable, and spread of variations but it will make them feel like they've "tamed" a disruption when, like climate change, they've done absolutely nothing.
A format where there’s an intro, middle sentence, middle sentence that is about the opposing view, concluding sentence. Written in a slightly persuasive non-combatant way.
Hmm. My posts on social media keep getting weighted down because it seems too much like ChatGPT. That means (laughing) I need to start writing in a very strange (weird) style so that I throw off the AI-detection algorithms.
Or gosh, students writing papers, and the false positives there... "Sorry kid, Turn-It-In says your paper was written by an AI, try again."
This was my finding as well. I abandoned this effort when I found that any created model would be inaccurate on small texts and could be circumvented with a bit of prompt engineering.