I had done the same for classifying and generating bookmarks of thousands of datasheets, along with a very naive yolo-based classificator (to detect pages made out of diagrams and pictures mostly).
Done with GLM-OCR, I had to watch text sloooowly crawl out of the llm and still have to live with hallucinations and the model not following the schema
The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ?
If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads.
I built a browser extension that does this, well for posts on twitter, hackernews, reddit etc.
If you want it for all text, it would also be feasible. I use a quantized mini-LM model that runs very fast and classifies eg your whole twitter feed in a couple of seconds.
Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
Images, absolutely, there are tell-tale artifacts from today's generators that simply aren't emitted by "natural" paths to create them, and you can "detect AI" with high confidence (for now). Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
With sufficient information you can derive a signal even in the presence of overwhelming noise. Assuming the noise is not perfectly correlated with the signal this is always possible.
Schemes like GPS, CDMA and DSSS are based upon this concept. GPS in particular is quite impressive in its ability to recover information that is received below the thermal noise floor.
If you have access to the detector, you can formulate a generative solution that avoids being flagged. Which gets me wondering why don’t model providers do that? There must be something about that that destroys semantic weights somehow.
The article discusses a technique by which the author achieves high accuracy at detecting AI written text. Unless you have a problem with their experimental method, this is the opposite of tarot card reading.
> we are well into undetectable sophistication with today's models
The article directly contradicts this, as do you, in your previous paragraph: "Sure you might be able to detect today's tells". The article is literally about a technique that detects today's tells.
Your comment is mostly expressing doubt that this technique will work reliably in the future, but it's framed as opposition to the article, which it's not: the article is about detecting today's AI-written text, at which it seems to be quite successful.
> Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
This is simply untrue, and completely divorced from reality.
Tarot card readings have literally zero predictive success. Last I checked, LLM-detection had a +90% success.
> Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it.
This does not sit well with personal experience and I wonder if it is just one of these questions of AI people being unaware of the level of skill that exists in domains they think have been automated.
It is of course possible that my tendency to spot LLM-written text has much to do with the way that it sounds like an averaged Californian college student to my British grammar-school-educated ears, as so many of the situations where I am encountering AI text are Brits using it without apparently realising they are giving themselves away.
But I know people who don't have particular technical skills in this sphere or a grammar-school background who also have an uncanny knack for pointing out LLM-written text.
> Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
I especially don't think this is true. Will they be able to do it in the future? Maybe. Is it possible to prompt a current cloud LLM to write in a way that is obvious? Yeah. (IMO Gemma 4 writes less detectably than most of them!)
But my instinct is that someone with any facility for language is going to be better than chance at spotting LLM-written text once it is three or four paragraphs long. So I think it should be possible in principle to train machine learning systems to detect those patterns.
The problems are simply too great if an LLM detector has any false positives at all. Imagine how soul-crushing writing an entire dissertation by hand and having it rejected because some “good enough” LLM detector decides you write too much like an AI.
We can measure false positive rate. The detector in the arricle is 85% accurate (not sure about false positives, but let's assume) which is too low to make conclusions, but enough when browsing the web and skipping reading likely-slop withiut accusing anyone.
If the false positive rate becomes <1% then it's better. The alternative is the world drowning under slop so I'd rather have imperfect detectors and have users aware they may fail in rare cases to avoid witch hunts. The general issue is that people only realize they're reading slop halfway through which is frustrating. If you know it from the start thanks to a detector and move on without commenting, no time waste, no frustration, less negativity towards LLM users.
I think the fundamental problem is that training current SOTA AI models is very expensive. If a simple "classical" model can detect them, presumably at much lower algorithmic cost, then why wouldn't the model trainers use these same tools to feed back into their models to improve them at low cost to make them better? It's an arms race. Any cheap pattern can and presumably will be used to retrain if it becomes and effective way to catch AI.
Could also be a problem of the form of P=NP. Validating might be very easy, but writing might be hard. Like the traveling salesman problem. It’s very easy to tell whether a specific path takes N units of time, but it’s hard to figure out if there’s any path, among all possible paths, that takes N units of time.
Because model providers are not optimizing for being indistinguishable from human text, and in fact, there is more value/demand in modeling a different distribution (ie an “agent” capable of producing vast amounts of concrete procedural/planning text interspersed) than there is in modeling the way humans write (ie GPT3).
Also you have to keep in mind that most AI companies are in fact trying to create and offer legitimate products and services to customers doing actually-useful work. They’re not trying to help fly by night hustlers scam people out of crypto or run spam campaigns, and in fact often voluntarily watermark to prevent misuse of their products.
You could argue that’s “just to avoid bad PR” and maybe you’re right, but that’s just another way of saying that it’s more profitable to prioritize other use cases than the deepfake/spam market. Spammers and fraudsters are shitty customers and a major brand risk.
I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing details. That needs effort, with or without AI.
there is not much point in detecting LLM generated text, in that humans are useing info from LLM's, but obfusicting it's origin, with there own garble, along with purely human garble, and almost(but not quite) human LLM product meaning that the threshold for rejecting "data" must be lowered, which personaly means a very very low tollerance for wierdness, except where it can yield imediate possitive cash flow
for the rest I do my own research and verification thank you very much
I could be wrong, but I just don’t see how trying to “detect” LLM generated texts is ever going to work. The only thing that makes any sense if you truly want to have confidence a human wrote it is some type of “proof of work“ system. I think there’s a lot of interesting ways to approach the proof of work problem with different pros and cons, but that is where our energy should be focused if we seriously want to solve this problem.
Am I the only who largely enjoys the output of LLMs more than most stuff written by humans? I find myself coming back to old chats with ChatGPT frequently because the output is amazing.
Neat. I will implement something like this for myself. I just need to reduce the spam a little. Imperfection is okay for a social network context like HN.
> Eventually, I faked my way through the thesis, and life moved on.
This is a very startling admission! I checked the Chinese (original?) version of the post, and saw the author uses the word "糊弄" (in the place of "faked"); I'm not a native speaker but I think this may come across more as a self-effacing comment on the low quality and/or effort behind their thesis, whereas the English version implies fraud. May be wise to change this!
Small encoder-only transformers are excellent at classifying LLM-Generated Text. I built an on-device iOS app using a custom small encoder that achieves an AUROC of 99.81 on RAID-bench.
I don’t think it actually matters (and it’s a losing strategy as others have noted).
This issue with AI generated stuff is that that it’s sometimes asymmetric: either the author worked very little to produce a lot of slop and now the reader(s) all have to do the heavy effort of reading it OR the author puts a little extra work in once and resolves all future readers’ burden.
If it was possible to boil down an artifact into a prompt + some resources that would be an interesting tool, or at least some way to tell if some artifact is “worth my time to read”
The thing I find most encouraging is that the best AI detector is still humans. Don't write the Turing test off yet.
From what I understand, your approach is clever, it's like an accent detector. Known models tend toward a specific median approach. Humans have a much richer degree of randomness. Riffing on Anna Karenina... All models are alike in that they present predictable patterns. Humans inevitably write in unique ways.
I gave a lot of thought to the idea that humans will devolve to the median led by volume of AI interactions, but in the end, I think we're still interacting with each other when not at work/on machines, and the fact that we even have a genetic heritage is always going to differentiate us.
> Sounds promising, right? I spent some time trying [perplexity], but results were disappointing—plenty of false positives and false negatives, and no reasonable threshold could be set.
Perplexity was widely considered SOTA in 2022. One part of it is because everyone was evaluating on open models or closed models that were still close (i.e. GPT-2 vs. GPT-3.5). Today, the gap is so much wider between the models you can use to compute perplexity and the frontier models people actually use.
Also so many AI text detection papers used a strawman RoBERTa baseline that was very undertrained for the task.
The synthetic mirrors method for data generation used here is the same as what we use at Pangram. Good blog post, thank you for sharing!
It looks like the text that this classifies might be Chinese, is that right? Do Chinese speakers have the same cultural aversion to AI-generated text? I’m wondering if there might be a different level of effort put into making text seem human-generated in English v. Chinese.
My theory is that the labs are RL'ing models to output easily classifiable text so they can avoid model collapse when training the models on data scraped from the web.
Of course, you can skew the distribution with some effort and generate text that avoids even the best classifiers out there (like Pangram), but even tech-savvy people aren't usually doing it (see the amount of AI-written posts that end up in HN and get tons of comments complaining about AI mannerisms), so I guess they're successfully avoiding like 99% of the slop using such classifiers.
I don't think it's in the interest of the labs to allow you to generate text that's indistinguishable from human prose. Especially since nobody would pay $1,000/mo just to generate text - but would do so for tasks like coding.
44 comments
[ 2.5 ms ] story [ 56.8 ms ] threadTomorrow, the LLMs will be training the humans thought patterns that will directly start skewing their natural writing.
Generation alpha is going to have a lot of trouble if we keep perpetuating the myth that you can really interpret text in an ongoing fashion.
Done with GLM-OCR, I had to watch text sloooowly crawl out of the llm and still have to live with hallucinations and the model not following the schema
If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads.
(the article was a good read, thanks!)
Check it out: https://slopsieve.com/extension
Accuracy is also much higher than this approach here. 0.9944 AUC, 0.966 acc@.5, 0.971 F1@.5
Images, absolutely, there are tell-tale artifacts from today's generators that simply aren't emitted by "natural" paths to create them, and you can "detect AI" with high confidence (for now). Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
Most people's issue with AI-generated llmish however is not that it's AI-generated. It's its insufferable tone.
So if we get to a point where we have to read tea leaves (an image you seem to appreciate) to determine if it's llmish or not, we'll have won by then.
Really: it's that full-on asshole tone I (and many others) want to see disappear from blogs, comments, LinkedIn, etc.
Schemes like GPS, CDMA and DSSS are based upon this concept. GPS in particular is quite impressive in its ability to recover information that is received below the thermal noise floor.
You can absolutely still tell.
> we are well into undetectable sophistication with today's models
The article directly contradicts this, as do you, in your previous paragraph: "Sure you might be able to detect today's tells". The article is literally about a technique that detects today's tells.
Your comment is mostly expressing doubt that this technique will work reliably in the future, but it's framed as opposition to the article, which it's not: the article is about detecting today's AI-written text, at which it seems to be quite successful.
This is simply untrue, and completely divorced from reality.
Tarot card readings have literally zero predictive success. Last I checked, LLM-detection had a +90% success.
This does not sit well with personal experience and I wonder if it is just one of these questions of AI people being unaware of the level of skill that exists in domains they think have been automated.
It is of course possible that my tendency to spot LLM-written text has much to do with the way that it sounds like an averaged Californian college student to my British grammar-school-educated ears, as so many of the situations where I am encountering AI text are Brits using it without apparently realising they are giving themselves away.
But I know people who don't have particular technical skills in this sphere or a grammar-school background who also have an uncanny knack for pointing out LLM-written text.
> Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
I especially don't think this is true. Will they be able to do it in the future? Maybe. Is it possible to prompt a current cloud LLM to write in a way that is obvious? Yeah. (IMO Gemma 4 writes less detectably than most of them!)
But my instinct is that someone with any facility for language is going to be better than chance at spotting LLM-written text once it is three or four paragraphs long. So I think it should be possible in principle to train machine learning systems to detect those patterns.
If the false positive rate becomes <1% then it's better. The alternative is the world drowning under slop so I'd rather have imperfect detectors and have users aware they may fail in rare cases to avoid witch hunts. The general issue is that people only realize they're reading slop halfway through which is frustrating. If you know it from the start thanks to a detector and move on without commenting, no time waste, no frustration, less negativity towards LLM users.
Also you have to keep in mind that most AI companies are in fact trying to create and offer legitimate products and services to customers doing actually-useful work. They’re not trying to help fly by night hustlers scam people out of crypto or run spam campaigns, and in fact often voluntarily watermark to prevent misuse of their products.
You could argue that’s “just to avoid bad PR” and maybe you’re right, but that’s just another way of saying that it’s more profitable to prioritize other use cases than the deepfake/spam market. Spammers and fraudsters are shitty customers and a major brand risk.
... if the assumption is LLMs are being optimized to evade such detection.
PS: I didn't read the actual article.
I think it is instruction-tuning that is having LLMs write differently from humans, and this is not being optimized away.
This is a very startling admission! I checked the Chinese (original?) version of the post, and saw the author uses the word "糊弄" (in the place of "faked"); I'm not a native speaker but I think this may come across more as a self-effacing comment on the low quality and/or effort behind their thesis, whereas the English version implies fraud. May be wise to change this!
This issue with AI generated stuff is that that it’s sometimes asymmetric: either the author worked very little to produce a lot of slop and now the reader(s) all have to do the heavy effort of reading it OR the author puts a little extra work in once and resolves all future readers’ burden.
If it was possible to boil down an artifact into a prompt + some resources that would be an interesting tool, or at least some way to tell if some artifact is “worth my time to read”
From what I understand, your approach is clever, it's like an accent detector. Known models tend toward a specific median approach. Humans have a much richer degree of randomness. Riffing on Anna Karenina... All models are alike in that they present predictable patterns. Humans inevitably write in unique ways.
I gave a lot of thought to the idea that humans will devolve to the median led by volume of AI interactions, but in the end, I think we're still interacting with each other when not at work/on machines, and the fact that we even have a genetic heritage is always going to differentiate us.
Perplexity was widely considered SOTA in 2022. One part of it is because everyone was evaluating on open models or closed models that were still close (i.e. GPT-2 vs. GPT-3.5). Today, the gap is so much wider between the models you can use to compute perplexity and the frontier models people actually use.
Also so many AI text detection papers used a strawman RoBERTa baseline that was very undertrained for the task.
The synthetic mirrors method for data generation used here is the same as what we use at Pangram. Good blog post, thank you for sharing!
Of course, you can skew the distribution with some effort and generate text that avoids even the best classifiers out there (like Pangram), but even tech-savvy people aren't usually doing it (see the amount of AI-written posts that end up in HN and get tons of comments complaining about AI mannerisms), so I guess they're successfully avoiding like 99% of the slop using such classifiers.
I don't think it's in the interest of the labs to allow you to generate text that's indistinguishable from human prose. Especially since nobody would pay $1,000/mo just to generate text - but would do so for tasks like coding.