I'm not late if people constantly put effort into finding LLM text, or every other comment on hacker news is either about something being LLM generator.
I had done the same for classifying and generating bookmarks of thousands of datasheets, along with a very naive yolo-based classificator (to detect pages made out of diagrams and pictures mostly).
Done with GLM-OCR, I had to watch text sloooowly crawl out of the llm and still have to live with hallucinations and the model not following the schema
The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ?
If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads.
Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
Images, absolutely, there are tell-tale artifacts from today's generators that simply aren't emitted by "natural" paths to create them, and you can "detect AI" with high confidence (for now). Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
> but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
Hard disagree. LLMs (especially base ones, that only received pre-training) can produce output that is undistinguishable from human writing (because that's what they were trained to do).
But commercial chat models are specifically tuned in a way that maximizes user engagement.
It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either.
And I don't think that's going to change anytime soon, unless their incentives change.
(We can say exactly the same thing about man-made stuff optimized for a specific purpose, like stock photography, clickbait titles or industrial food: they aren't stereotypical because their creator lacks the skill to make them otherwise, they are like that because that's what works best).
They're also designed to not offend anybody, so their output tends to be very bland even compared to the most milquetoast of human beings. I was only surprised once when ChatGPT responded with an enthusiastic "hell yes" seemingly organically, but 99.9% of the time these AI services clearly are instructed and trained to provide flavorless word vomit. I don't think there's a technical reason why an LLM couldn't produce totally convincing output, but internet grifters don't need to go through that trouble. It's like how most phone, email, and social media scams come off as completely transparent to most of us, but that's the whole point; we're not the target audience of the scams. Readers looking for substance, nuance, and real opinions aren't going to notice if something with written by an LLM – unless there are some cliche punctuation tells.
When DANmode bypasses were a common thing the LLMs would drift significantly far from corporate speak.
But that's the point of corporate speak, you tend not to say thing that may offend your clients and deprive the company of future revenue. Of course there are some companies that make their living being 'counter-culture' and saying what they want, but they are a small percentage of all revenue.
> But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either.
There are two problems with this.
The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".
And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.
> The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".
The thing is, humans are significantly worse at maximizing numerical goals than computers.
> And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.
Anyone can already do that right now, just grab unsloth studio and fine-tune your local Gemma, but nobody cares. People posting slop content don't care if pangram or I flag their slop with certainty, they are using the easiest option, which is commercial chat models. And given this segment of user doesn't care, the provider have zero incentive to provide a dedicated stealth model for that purpose.
I mean, back when I was spam filtering setting up a simple Bayesian classifier was easy. Train it on your spam and ham and it worked damned good. "Mission Accomplished".... until it wasn't. Spam rates started climbing and it started getting harder than ever to filter them.
There is always an incentive to get spam to bypass filters, so as your filters increase in accuracy, those attempting to pass said filters adjust their behaviors.
Spammers/cheaters/whateverers will at least just use a second pass filter that uses one of these 'ai scoring' systems to beat said AI scoring systems. So while it's worthwhile to do it at this moment, this window will rapidly close.
I don't think it's a very good remark, as there's significantly less email spam than 20 years ago.
Another example is ad-blocker-blocker. There was a little bit of an arm race between ad blockers and advertisers in the middle of the 2010s, but it didn't last long. Advertisers mostly just decided not to care about ad-blockers.
>Advertisers mostly just decided not to care about ad-blockers.
Directly not to care because they lost in court.
And yet the biggest advertizer on Earth (Google) decided to change their browser to make adblocking far more difficult. That or they say "just use an app, oh and turn on notifications". I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.
There is significantly more spam than 20 years ago, just less of it reaches your inbox. This is a very important distinction as the cost of spam filtering is just as high as ever. On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them. This allows these companies to have an overwhelming influence on email on the internet, to the point they can send spam with near impunity, and where if your system does it will be nuked from orbit by their systems.
And much like now Google supplies both the email spam, and the solution to the spam, they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.
> Directly not to care because they lost in court.
I'm pretty sure the illegal sport streaming websites didn't stop doing that just because it became illegal, otherwise they could have stopped their activity altogether while they were at it…
> I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.
I, at least temporarily, won for the past decade.
> nd yet the biggest advertizer on Earth (Google) decided
This is actually an argument in my direction! The owners of websites (which are also the ones posting slop today) didn't care enough and the situation only changed because Google moved.
I expect the same thing with slop. Individual websites won't make any effort to make their slop unblockable, and it will only be a problem if OpenAI/Anthropic/Google decide that they care about this market. But unlike Google in the ads market, I don't think the model providers have any reason to care. The web is already dead in their mind anyway.
> There is significantly more spam than 20 years ago, just less of it reaches your inbox.
This goes against your very argument from earlier!
> On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them.
Out of convenience, but you don't need that to be practically free of spam. Whatever version of SpamAssassin is being run on OVH's mail servers has been enough for that purpose for me.
> they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.
Again, they don't care about the web. They just crawl it for content but they don't want you to read any webpage, they want you to stay in their chatbot. Every other use-case is nonexistent to them (except coding agents, of course, but that's a different product altogether).
i think one thing overlooked by this perspective is that many of a detectors adversaries are not that sophisticated. so despite this i think it is a useful thing to try to do. particularly when people are trying to do fraud which will often having to use abliterated models and generally trying to be as economical in their efforts
Sure it is; we do it all the time, and then we modify each other's etc, etc; english we speak today was spoke yesterday waspake the same in yesteryears; we have no trouble dating english or other languages to a time.
A better argument is people themselves are just too influenced by reading that they'll sound like LLMs in a couple of years.
There are two problems, false positives and changing the LLM's pattern.
It's really easy to have a false positive and false positives can be very harmful if the person using the detector isn't aware of that risk.
It's also very easy to change the pattern of LLM output. You can provide basic prompting that will significantly change the structure of the output. For example, having it utilize the Wikipedia article on signs of AI writing and avoid everything it describes. https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
Not really. The false positives for the SOTA detector are very very low.
"It's also very easy to change the pattern of LLM output."
Not in a way that can reliably avoid detection. The problem is the patterns are baked into the distribution itself. It's smoothed over, so it becomes difficult to prompt your way out of that.
Signal is always easier to detect with more data to work with. Largely AI generated books are a vastly different situation than a one paragraph homework assignment. But multiple rounds of homework assignments would change the accuracy.
Whether a text was written by a human or not is just a single bit of information. So you can't rule out its detectability a priori, since even the shortest text contains more information than that.
As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent to "This looks like spam; no human in their right mind would write this spam by hand if they could get an LLM to write it, therefore it's most likely written by an LLM."
You definitely can rule out the general case a priori. If the problem were possible, for every text there would be a unique provenance label “human” or “ai”. But since humans and machines have both written many texts, it is not possible.
As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the labels conflict for the same piece of text.
The data is statistically inseparable which makes it impossible to classify from text alone.
That's like saying whether or not you're going to fall in love this year is just one bit of information, so you might be able to read it from astrology. Yeah, sure, it might happen for some people with a certain star sign. But across the population there is zero reason to believe that there is a) any significant correlation and b) enough data variation in to even distinguish classes of humans.
"Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it...it's a bad fiction to perpetuate that any of this is anything more than tarot card reading."
Not true at all. Pangram is highly effective and has a very low false positive rate.
The post here is impressive for a small project, it looks like they independently thought of one of the core ideas Pangram uses of creating twins to compare.
So, if the decision from Pangram determined, on every assignment, if you would be expelled from university for plagiarism, would that be acceptable to you regardless of how you actually did the work?
If you would not be okay with that, what level of consequence would be acceptable for the output from this tool?
I’d want detectors to be as accurate as possible, false positives of 1 in 10000 seems like a good starting point. I believe their results have been independently tested.
And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process.
The reason I asked is that I have a hard time understanding the point of these tools. When it comes to education, it can be a matter of learning objectives. But outside that, what's the point?
The prediction from the tool is pointless for deciding on copyright or contract issues, and other text should be judged on its correctness or applicability to the task.
If all the tool is good for is "maybe this student cheated, but only an in-depth investigation would maybe prove it", it isn't a very useful tool, because it's more straightforward to just mandate that evidence is submitted regardless of what the tool says. On top of that, even the lack of evidence of manual work isn't good proof of using LLMs.
I don't know, the thing about most text slop is how little effort goes into disguising it (for now, anyway). I'm sure anyone dedicated can go undetected, but it's the really low-effort stuff that's generally the problem. If you can catch some of it, that's something at least.
The problems are simply too great if an LLM detector has any false positives at all. Imagine how soul-crushing writing an entire dissertation by hand and having it rejected because some “good enough” LLM detector decides you write too much like an AI.
As I recall, a few years ago (in the era of first generation LLMs), a professor in Texas used an anti-plagiarism tool that flagged more than one-third of the class using AI in an exam, and used that finding to give them a failing grade.
If memory serves, one student objected strenously and ran the professor's own work (published 10 years earlier) into the same tool and it flagged that work as AI-generated.
Exactly. The more corporate and proper you tend to speak, the more likely it's to classify you as an LLM. It's like the classifiers want us to talk like trash at their current rate. This seems to be really problematic for ESL speakers/typers that may have been trained on a smaller, more proper subset of the language.
I think the fundamental problem is that training current SOTA AI models is very expensive. If a simple "classical" model can detect them, presumably at much lower algorithmic cost, then why wouldn't the model trainers use these same tools to feed back into their models to improve them at low cost to make them better? It's an arms race. Any cheap pattern can and presumably will be used to retrain if it becomes and effective way to catch AI.
It’s simply not a priority. The labs can do many things. Making text non-LLM is not really that useful. Analogous to Facebook not picking up the obvious $20 bill in front of them. It’s because they’ve got $100 bills at their feet they’re picking up.
Not a priority currently. Selling services to spammers... I mean marketers is still big money and eventually someone will pick it up. If training costs ever drop, then it's one of the first things that will happen.
In part because model vendors specifically prefer when people think that lots of content is produced by their model. The more Claude-like writing appears on the internet, the more signal there is to investors that people are using Claude for a greater number tasks.
I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing details. That needs effort, with or without AI.
The easiest way is to keep track of the text's edit history, keeping a block of edits over time and having them signed by a timestamp authority. The final edit history can then be inspected by some external authority, then signed if the edit history looks human. I have a blog post from 2023 on this topic: https://helbl.ing/Written-Proof-of-Work/
For Google Doc users, you can already inspect the edit history over time to verify that text is written by a human.
That human might have used AI. You can never know. Hand fixed AI output, human just polished the corners? Light rewording of a full text written by hand, because the author is not confident in their writing? Actual human text, but after researching with AI?
I am working on a browser extension to help with that. Basically it interposes on any text field and canvas and if user pastes a large amount of text (copied form example from a chat bot), the extension will "replay" that text at normal, human-editing pace, and introduce typos that are fixed through later edits.
As a verbal processor, I hate this. I use whisper v3 large all the time from voice memo dumps that are AI grammar corrected and this would flag when I copy paste it in :/
Sufficiently advanced AI use is probably fine. The slop everyone complains about has certain tells specifically due to some combination of the following:
- The author is conducting some kind of hustle.
- The author doesn't bother editing.
- The author lacks the taste and awareness enough to see it looks.
- The author thinks you, the reader, lack taste and awareness.
- The author is using it as a kind of smoke bomb to get rid of you.
In such cases, nothing is done about the LLM's distinctive "voice". It dominates the text and it's easy to detect. It stands as a signifier of the above, even if it's otherwise not intrinsically a problem to use AI.
…All I know is that sometimes I will read e.g. a HN submission, and it becomes pretty obvious partway through that the article was AI generated.
If I can do it, an algorithm should be able to do it. Maybe in the the models will get so good that it is literally impossible to differentiate human vs computer authorship, but that’s obviously not the case today.
I've noticed there seems to be a default style that is easier to detect. I've noticed it harder to detect when asking an LLM to use a different style (more conversational, avoid sounding like an AI, don't use emdashes, etc). I wonder if that's what you're picking up too - the instances where people make no effort to change the style of the output.
No idea, I'm not convinced it matters that much? Like, if people are using AI and I legitimately can't tell at all (and I'm not their teacher or something)... okay, fair.
What's likely to happen though, is human idiomatic writing will degrade to AI level and the two will converge. Just like nowadays it is harder to tell if some (human) non-native English speaker wrote some English text online whereas 20 years ago it was very obvious.
there is not much point in detecting LLM generated text, in that humans are useing info from LLM's, but obfusicting it's origin, with there own garble, along with purely human garble, and almost(but not quite) human LLM product meaning that the threshold for rejecting "data" must be lowered, which personaly means a very very low tollerance for wierdness, except where it can yield imediate possitive cash flow
for the rest I do my own research and verification thank you very much
I could be wrong, but I just don’t see how trying to detect LLM generated texts is ever going to work. The only thing that makes any sense if you truly want to have confidence a a human wrote it is some type of “proof of work“ system. I think there’s a lot of interesting ways to approach the proof of work problem with different pros and cons, but that is where our energy should be focused if we seriously want to solve this problem.
Am I the only who largely enjoys the output of LLMs more than most stuff written by humans? I find myself coming back to old chats with ChatGPT frequently because the output is amazing.
Neat. I will implement something like this for myself. I just need to reduce the spam a little. Imperfection is okay for a social network context like HN.
> Eventually, I faked my way through the thesis, and life moved on.
This is a very startling admission! I checked the Chinese (original?) version of the post, and saw the author uses the word "糊弄" (in the place of "faked"); I'm not a native speaker but I think this may come across more as a self-effacing comment on the low quality and/or effort behind their thesis, whereas the English version implies fraud. May be wise to change this!
Well cheated would definitely imply fraud. “Faking it” as in “fake it till you make it” is more like pretending you know about a topic until you learn enough on the job to participate competently.
I don't know if the Chinese text implies something different, but I think even in English it's pretty normal for people to claim they 'faked' their way through something without referring to fraud.
E.g. "I faked my way through the interview!" = "I did my best to respond to questions I did not feel fully prepared for, and managed to get through the interview"
Small encoder-only transformers are excellent at classifying LLM-Generated Text. I built an on-device iOS app using a custom small encoder that achieves an AUROC of 99.81 on RAID-bench.
I don’t think it actually matters (and it’s a losing strategy as others have noted).
This issue with AI generated stuff is that that it’s sometimes asymmetric: either the author worked very little to produce a lot of slop and now the reader(s) all have to do the heavy effort of reading it OR the author puts a little extra work in once and resolves all future readers’ burden.
If it was possible to boil down an artifact into a prompt + some resources that would be an interesting tool, or at least some way to tell if some artifact is “worth my time to read”
78 comments
[ 3.0 ms ] story [ 35.3 ms ] threadTomorrow, the LLMs will be training the humans thought patterns that will directly start skewing their natural writing.
Generation alpha is going to have a lot of trouble if we keep perpetuating the myth that you can really interpret text in an ongoing fashion.
https://www.washingtonpost.com/opinions/2025/08/20/chatgpt-c...
Done with GLM-OCR, I had to watch text sloooowly crawl out of the llm and still have to live with hallucinations and the model not following the schema
If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads.
The link you sent is for generating text which attempts to defeat those classifiers.
Images, absolutely, there are tell-tale artifacts from today's generators that simply aren't emitted by "natural" paths to create them, and you can "detect AI" with high confidence (for now). Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
Hard disagree. LLMs (especially base ones, that only received pre-training) can produce output that is undistinguishable from human writing (because that's what they were trained to do).
But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. And I don't think that's going to change anytime soon, unless their incentives change.
(We can say exactly the same thing about man-made stuff optimized for a specific purpose, like stock photography, clickbait titles or industrial food: they aren't stereotypical because their creator lacks the skill to make them otherwise, they are like that because that's what works best).
But that's the point of corporate speak, you tend not to say thing that may offend your clients and deprive the company of future revenue. Of course there are some companies that make their living being 'counter-culture' and saying what they want, but they are a small percentage of all revenue.
Did you actually try them? I did.They generated even more "slopey" text than instruction-tuned ones.
There are two problems with this.
The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".
And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.
The thing is, humans are significantly worse at maximizing numerical goals than computers.
> And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.
Anyone can already do that right now, just grab unsloth studio and fine-tune your local Gemma, but nobody cares. People posting slop content don't care if pangram or I flag their slop with certainty, they are using the easiest option, which is commercial chat models. And given this segment of user doesn't care, the provider have zero incentive to provide a dedicated stealth model for that purpose.
There is always an incentive to get spam to bypass filters, so as your filters increase in accuracy, those attempting to pass said filters adjust their behaviors.
Spammers/cheaters/whateverers will at least just use a second pass filter that uses one of these 'ai scoring' systems to beat said AI scoring systems. So while it's worthwhile to do it at this moment, this window will rapidly close.
Another example is ad-blocker-blocker. There was a little bit of an arm race between ad blockers and advertisers in the middle of the 2010s, but it didn't last long. Advertisers mostly just decided not to care about ad-blockers.
Directly not to care because they lost in court.
And yet the biggest advertizer on Earth (Google) decided to change their browser to make adblocking far more difficult. That or they say "just use an app, oh and turn on notifications". I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.
There is significantly more spam than 20 years ago, just less of it reaches your inbox. This is a very important distinction as the cost of spam filtering is just as high as ever. On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them. This allows these companies to have an overwhelming influence on email on the internet, to the point they can send spam with near impunity, and where if your system does it will be nuked from orbit by their systems.
And much like now Google supplies both the email spam, and the solution to the spam, they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.
I'm pretty sure the illegal sport streaming websites didn't stop doing that just because it became illegal, otherwise they could have stopped their activity altogether while they were at it…
> I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.
I, at least temporarily, won for the past decade.
> nd yet the biggest advertizer on Earth (Google) decided
This is actually an argument in my direction! The owners of websites (which are also the ones posting slop today) didn't care enough and the situation only changed because Google moved.
I expect the same thing with slop. Individual websites won't make any effort to make their slop unblockable, and it will only be a problem if OpenAI/Anthropic/Google decide that they care about this market. But unlike Google in the ads market, I don't think the model providers have any reason to care. The web is already dead in their mind anyway.
> There is significantly more spam than 20 years ago, just less of it reaches your inbox.
This goes against your very argument from earlier!
> On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them.
Out of convenience, but you don't need that to be practically free of spam. Whatever version of SpamAssassin is being run on OVH's mail servers has been enough for that purpose for me.
> they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.
Again, they don't care about the web. They just crawl it for content but they don't want you to read any webpage, they want you to stay in their chatbot. Every other use-case is nonexistent to them (except coding agents, of course, but that's a different product altogether).
A better argument is people themselves are just too influenced by reading that they'll sound like LLMs in a couple of years.
It's really easy to have a false positive and false positives can be very harmful if the person using the detector isn't aware of that risk.
It's also very easy to change the pattern of LLM output. You can provide basic prompting that will significantly change the structure of the output. For example, having it utilize the Wikipedia article on signs of AI writing and avoid everything it describes. https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
Not really. The false positives for the SOTA detector are very very low.
"It's also very easy to change the pattern of LLM output."
Not in a way that can reliably avoid detection. The problem is the patterns are baked into the distribution itself. It's smoothed over, so it becomes difficult to prompt your way out of that.
As long as LLMs are used to write texts humans wouldn't want to write if they could help it (that's why they're getting an LLM to do it, after all), they'll remain detectable. Even if the reasoning might end up equivalent to "This looks like spam; no human in their right mind would write this spam by hand if they could get an LLM to write it, therefore it's most likely written by an LLM."
As an example, you could imagine a giant lookup table that deterministically mapped every text ever written to “human” or “AI”. You would very quickly run into situations where the labels conflict for the same piece of text.
The data is statistically inseparable which makes it impossible to classify from text alone.
Not true at all. Pangram is highly effective and has a very low false positive rate.
The post here is impressive for a small project, it looks like they independently thought of one of the core ideas Pangram uses of creating twins to compare.
You can see how it works here: https://arxiv.org/pdf/2402.14873
If you would not be okay with that, what level of consequence would be acceptable for the output from this tool?
I’d want detectors to be as accurate as possible, false positives of 1 in 10000 seems like a good starting point. I believe their results have been independently tested.
And as a separate matter, any tool for evaluating students should be applied fairly, safely, and with adequate human review and due process.
You need good tools and good oversight.
The reason I asked is that I have a hard time understanding the point of these tools. When it comes to education, it can be a matter of learning objectives. But outside that, what's the point?
The prediction from the tool is pointless for deciding on copyright or contract issues, and other text should be judged on its correctness or applicability to the task.
If all the tool is good for is "maybe this student cheated, but only an in-depth investigation would maybe prove it", it isn't a very useful tool, because it's more straightforward to just mandate that evidence is submitted regardless of what the tool says. On top of that, even the lack of evidence of manual work isn't good proof of using LLMs.
Most people's issue with AI-generated llmish however is not that it's AI-generated. It's its insufferable tone.
So if we get to a point where we have to read tea leaves (an image you seem to appreciate) to determine if it's llmish or not, we'll have won by then.
Really: it's that full-on asshole tone I (and many others) want to see disappear from blogs, comments, LinkedIn, etc.
If memory serves, one student objected strenously and ran the professor's own work (published 10 years earlier) into the same tool and it flagged that work as AI-generated.
EDIT: HN item from June 2023 https://news.ycombinator.com/item?id=36215823
For Google Doc users, you can already inspect the edit history over time to verify that text is written by a human.
I wish there was a solution
- The author is conducting some kind of hustle.
- The author doesn't bother editing.
- The author lacks the taste and awareness enough to see it looks.
- The author thinks you, the reader, lack taste and awareness.
- The author is using it as a kind of smoke bomb to get rid of you.
In such cases, nothing is done about the LLM's distinctive "voice". It dominates the text and it's easy to detect. It stands as a signifier of the above, even if it's otherwise not intrinsically a problem to use AI.
If I can do it, an algorithm should be able to do it. Maybe in the the models will get so good that it is literally impossible to differentiate human vs computer authorship, but that’s obviously not the case today.
He literally demonstrated a working system in this post. Do you mean you'll never get to 100% accuracy? Clearly, but you don't need that.
This is a very startling admission! I checked the Chinese (original?) version of the post, and saw the author uses the word "糊弄" (in the place of "faked"); I'm not a native speaker but I think this may come across more as a self-effacing comment on the low quality and/or effort behind their thesis, whereas the English version implies fraud. May be wise to change this!
E.g. "I faked my way through the interview!" = "I did my best to respond to questions I did not feel fully prepared for, and managed to get through the interview"
This issue with AI generated stuff is that that it’s sometimes asymmetric: either the author worked very little to produce a lot of slop and now the reader(s) all have to do the heavy effort of reading it OR the author puts a little extra work in once and resolves all future readers’ burden.
If it was possible to boil down an artifact into a prompt + some resources that would be an interesting tool, or at least some way to tell if some artifact is “worth my time to read”