Previously, spam has been reported and filtered via "flag" and "report" buttons.
Why is that not sufficient for site operators to separate "AI Slop" from other articles?
If existing sites choose to tolerate "AI Slop" (why doesn't HN have this problem?), does that create a market opportunity for competitors with better filters?
Before LLMs, the cost to generate spam content was non-zero. Now, it's effectively zero. These infinite supplies of AI Slop should be handled differently and more aggressively that legacy spam.
Mostly because most spam isn't going to be reported. Traditional anti-spam works because once you have a handful of reports, you can probably generalize it to an entire class of spammy pages generated using a particular technique, and get rid of everything.
LLM-generated content is not distinguishable from human content by any simple rule, so you lose the ability to police the platform that way.
> Mostly because most spam isn't going to be reported. Traditional anti-spam works because once you have a handful of reports, you can probably generalize it to an entire class of spammy pages generated using a particular technique, and get rid of everything.
Tell that to Elon Musk. He seems to unable or unwilling to stop a certain type of fake female bot spam on X that has extremely simple to spot and unique patterns.
> Tell that to Elon Musk. He seems to unable or unwilling to stop a certain type of fake female bot spam on X that has extremely simple to spot and unique patterns.
It probably drives engagement, so there's no need to interfere. Perhaps it should be encouraged even..
A lot of the time, AI slop is difficult to discern from authentic content until you’re somewhere in the middle, at which point something starts to feel off about it. At least, that’s been my experience coming across AI-written Blender tutorials on YouTube. All this makes it difficult to quickly report as spam. There’s also a lot of plausible deniability since a lot of people have developed a hair-trigger response to writing they don’t like or find compelling and just immediately go “It’s AI!” even when it isn’t.
"A lot of the time, AI slop is difficult to discern from authentic content until you’re somewhere in the middle, at which point something starts to feel off about it."
And it's only going to get worse. I don't know what the limits of LLMs are, even before we start discussing future architectures and capabilities, but it certainly seems to me that if they just scale a bit farther on their current trajectory it is going to be very hard to detect AIs simply by the content.
One of the adaptations I've adopted is that if your content looks like it was generated from a simple prompt, no matter how long the resulting content, I just toss it. "Write me a beginner's tutorial for Python" isn't going to produce a document that the internet needs another copy of or that social media sites need to spend time linking to. Unfortunately that means even if the "tutorial" was human-generated, it's probably getting the axe too, unless it has a very interesting or unusual approach, but it isn't clear to me that that is actually a large problem.
Why isn't it? As far as I can remember, all the forums had lot of outright spam, even forums which are orders of magnitude less popular than HN. You can even see spam in usenet. Reddit/Discord is full of it. I wonder are dang/moderators the only reason for this?
I've seen a few comments on HN that looked likely to have been generated by LLMs, but they tend not to get any replies or up-votes so there's not much incentive for people to keep on trying to post them.
To be honest, I do use GenAI to help me format my comments, including this one. However, I put in substantial input by outlining my thoughts in detail, so I treat it more as a sophisticated grammar checker. As a non-native speaker, my writing is sufficient for business, but I don’t consider myself a strong writer. GenAI helps me convey my thoughts more clearly and efficiently, which has been tremendously valuable.
Please refine:
To be honest, I do use GenAI to help me format my comments including this one. I do, however put in substantial input outlining my thought in detail, so I use it more like a glorified grammar checker. It helps me a lot as I'm not a good writer (as being non-native, just sufficient to be able to thrive in business environment, just not a good writer. It helps me tremendously to get my thought across efficiently.
Perhaps I'm naive or old fashioned, but I prefer this unpolished version. You clearly have an unclosed parenthesis that's attached to a rough sentence, but two minutes of rephrasing will serve you well in improving both your writing and your speech. If you must use it, I'd suggest asking for suggestions rather than rewrites.
Writing is not just about syntax, but also about communicating who you are. This is your true voice.
Thank you—I understand where you're coming from. My interest in using GenAI stems from its ability to catch subtle implications or unintended double meanings in my writing. I've had plenty of "that's not what I meant" moments in the past, so using AI helps me align my phrasing with my true intent. It offers an objective lens, making it easier for me to convey my thoughts clearly, especially since I sometimes find it challenging to step out of my own subjective perspective.
Some roughness in my recent writing comes from being on mobile, which makes reviewing even harder! My writing has improved, though, even when I’m working without AI, likely from exposure to better writing flows. I’d say I used to rely more on edits (20% input to 80% edits), but now it’s closer to 75% input and 25% edits. Hopefully, I can refine it even further.
Fair enough. If you ever decide to do the proofreading yourself, you may find a prompt like this useful:
"Consider the following message. Distill the main thrust of the message, and list any implied statements made within. Be particularly sensitive to any negative connotations, and summarize the writer's feelings towards the reader:"
Medium was flooded with human-created slop before OpenAI got big. I work in Spark a lot, and basically every question I put into Google has a few lame Medium articles that basically recap the docs or have some trivial example code. Ironically, I turn to Claude now as my primary source. It's very good at Spark.
I have noticed that someone immediately calls out pretty much every bit of content as AI generated, whether it obviously is or not. I guess this is a best case scenario.
I do wonder what will spring up next to allow high quality content to be found.
That's more a consequence of "AI" becoming a synonym for "always bad" in creative spaces (and inversely "if it's bad, it must be AI-generated"), despite the fact that there's some nuance there.
You forget exactly who is complaining that AI is always bad, with no nuance. It's a bit like observing that in every book and movie writers have a very strong tendency to be rich superheroes with incredible morals and 50 admirers on every finger.
There's so many LLM bots on Twitter now, presumably trying to game the revenue sharing system. Some of them even seem to be getting fancy with multi-modal models because they try to respond to the content of images rather than just the text like they used to do.
True, I'm always making sure my Kindle is face down when I'm not using it so that my coworkers don't think I'm reading whatever insane AI slop Title Amazon happens to be advertising to me at the moment.
I really don't see how these people making the ebooks are making any money on these books either since they cost 99 cents and they pay for advertising. Maybe they're using llama2 idk lmao, either way I've seen a ton of embarrassing titles pop up on there and don't want people thinking I'm interested in that slop.
Yeah this is all very dystopian. I don't see a 'pay to publish' scheme working, neither do I see 'pay to read'. Maybe I could see 'pay to prove humanity' working, but the Voight-Kampff test is expensive, inaccurate and can be gamed, and accounts which passed it will be sold.
We're between a rock and a hard place with the internet's collective signal to noise ratio plummeting and no easy way to fix it.
Speaking of everywhere... My Google results are absolutely flooded with this shit now and it's driving me crazy. It won't be long before site searching (site:reddit.ciom/r/whatever, etc.) is similarly plagued. It's very frustrating!
SEO was annoying but there was still value in the results. Now, as soon as I detect or start to believe some result is AI bullshit I immediately close the tab out.
Human curation will become more and more valuable, perhaps precisely because it does not scale.
You know which travel advice I prefer? The one from a small, local blogger that loves their city. You know where I find my music? At house parties. More and more, I also crawl the profiles of individual users for content that I might like.
The cosy web is my refuge from scale and its consequences.
And that lack of scale means it has little mass monetization potential. It's actively hostile to weirdos just trying to make a buck. I don't get randos from who-knows-where trying to hawk their wares in my private group chats, so those recommendations actually carry weight.
I suspect more sites like wikipedia and the old bbs forums but ones maybe that require an actual ID step, which sucks but might be the only way. Or maybe something like Musk did but a one time $10 fee. Most bots won’t pay up to spread their crap and when they do their numbers should be gated enough by $10 that it won’t take much effort to kick them for abusing the system
one of the saddest parts of the whole thing is finding out how many fucking arseholes out there have been waiting to flood the internet with crap, and had previously only failed because they were too lazy to do all that typing.
It's an incentives problem, and the ad industry is largely to blame. It's the same cause as much of the destruction and centralization of the web, the rise of clickbait, rising extremism and political polarization, negativity-driven engagement, and myriad other public problems. An attention-drivin economy is disastrous for mental and public social health. And some people still defend the ad industry, even going so far to laud it as a public good, even talking about it as if it's a charity and not a very rich industry, funding a "free Internet".
We missed the boat long ago on dismantling or neutering the ad industry before it did massive irreparable damage. Whatever we do to fix this mess is still going to leave prominent scars for generations.
I don't see the clickbait / extremism promoting content as anything particularly new. In the UK the "red top" newspapers used to supply it and in the recent past it was Fox News.
It was always possible to cocoon yourself into news and opinion you agreed with.
AI slop does seem a little different, if only because it's machine generated and the sheer amount of it would not have been possible when rubbish was human generated and the gatekeepers of publishing were newspaper editors.
I've seen this on Substack as well. For example, I ran across this account on Substack is posting a lot of ChatGPT-generated articles to reddit. This is clearly ChatGPT-generated based on a very simple prompt:
> CEO Tony Stubblebine says it “doesn’t matter” as long as nobody reads it.
It seems like the leaders of all of these online platforms are going to just bury their heads in the sand and pretend that this massive problem isn't one.
This AI trend is absolutely going to destroy many (most?) of these companies. For better or worse, the new age of the internet is already here.
Maybe this is just a Band-Aid, but Medium has a "boost" mechanism.
I'm not sure of the exact machinations, but the times that my posts have been boosted, it's always been by a publication on Medium where a human editor has read my content and 1) asked that I submit to their publication and 2) submitted to Medium for boosting.
Seems like this type of filter -- as long as they actually keep humans in the loop -- can potentially work to keep high quality content more visible.
Why wouldn't they? They've spent the last decade pretending that the clickbots sustaining the internet ad model are no big deal and apparently the market agrees with them!
I'm a paying member of Medium although, due to time constraints, I only really read the articles in the Weekly Digest emails at the weekend.
I have noticed a lot of these articles that just make no sense at all starting to appear in my weekly email to the point that I probably won't renew next year if it stays this way, so it isn't just "nobody reading them".
Currently on ChatGPT three of the top picks on the explore page are "AI Humanizers" for making generated content appear like it was created by a human. Slop slop slop.
If anything good comes of AI slop, it will be stronger (and possibly more personalized) collaborative filtering. I wish I could upvote and downvote more content in more places in a way that my feedback would be used by people who have chosen to trust me (and vice versa).
Amongst the others things that walled gardens like Facebook, Instagram, Twitter, and TikTok have taken is the open availability of social signals to rank content. Previously Google used links to build some trust metric, but almost no real person creates public links anymore.
Is it particularly worse if it is AI slop? Slop has been flooding a lot of platforms before AI, AI slop makes it harder to detect if it is artificially generated but some of it is better than human slop.
I’m waiting for the point in time when content platforms no longer try to downplay their generated content but instead simply embrace it and even go as far as excluding human only content.
Like for example a TikTok, but all the videos are generated by the platform itself using likes/skips to drive its algorithm to generate more addicting content. That seems like the logical conclusion of where all of this is going.
They can then drop the creator side of the market and a lot of moderation issues that come with it, while gaining full ownership, control, and monetization of the underlying content.
Hey it's me, Max. I ran the analysis for WIRED and got them their initial 47% number for AI content.
The CEO accused me of trying to extort him because I sent a short email with our findings prior to the publication of this article.
He also casts doubt on the efficacy of AI detection (fair) but I think our AI detection model is orders of magnitude more accurate than the others and I stand by its predictions.
We benchmark on pre-2023 datasets of O(10M) documents not in our training set. Other detectors seem to have between 1-3% false positive rate and ours is around 1 in 10,000 as of our latest model update. We do a lot of active learning + core set selection to keep FPR low and improve recall on larger LLMs. Our white paper with some methodology is here: https://arxiv.org/abs/2402.14873
I noticed something, you achieved near 100% accuracy on most domains in every domain but scientific which made me wonder, how much is that could be due to how "strict" and "profrssional" these papers could be, or maybe how a slightly disproportionate number of the training data for these LLM could be from science based articles and papers, as they are generally viewed to be "high quality"
Interesting read either way, best of luck on your project (:
Provable quality human curation has always been valuable, but hard to identify. It's becoming a smaller slice of the pie as it fills up with AI slop. Good time for newsletters and podcasts to be out in force if the higher quality content can be identified.
I’ve come to simply realize the problem isn’t AIs, it’s humans.
More specifically, humans who feel the need to make a quick buck.
Actually, the problem isn’t those humans, it’s the societal structure creating incentives and pressures that lead to a large percentage of humans feeling like they need to make a quick buck. Whether that quick buck be for cost of living or for social status gain.
I'm sure there are thousands of resumes out there with a Medium blog URL on them, which probably impresses HR but not so much if you actually read them. It's all just another way to "build your personal brand".
"Earlier this year, WIRED asked AI detection startup Pangram Labs to analyze Medium. It took a sampling of 274,466 recent posts over a six-week period and estimated that over 47 percent were likely AI-generated. “This is a couple orders of magnitude more than what I see on the rest of the internet,” says Pangram CEO Max Spero. (The company’s analysis of one day of global news sites this summer found 7 percent as likely AI-generated.)"
75 comments
[ 6.6 ms ] story [ 133 ms ] threadWhy is that not sufficient for site operators to separate "AI Slop" from other articles?
If existing sites choose to tolerate "AI Slop" (why doesn't HN have this problem?), does that create a market opportunity for competitors with better filters?
LLM-generated content is not distinguishable from human content by any simple rule, so you lose the ability to police the platform that way.
Tell that to Elon Musk. He seems to unable or unwilling to stop a certain type of fake female bot spam on X that has extremely simple to spot and unique patterns.
It probably drives engagement, so there's no need to interfere. Perhaps it should be encouraged even..
> why doesn't HN have this problem?
HN comments aren’t easily monetizable.
And it's only going to get worse. I don't know what the limits of LLMs are, even before we start discussing future architectures and capabilities, but it certainly seems to me that if they just scale a bit farther on their current trajectory it is going to be very hard to detect AIs simply by the content.
One of the adaptations I've adopted is that if your content looks like it was generated from a simple prompt, no matter how long the resulting content, I just toss it. "Write me a beginner's tutorial for Python" isn't going to produce a document that the internet needs another copy of or that social media sites need to spend time linking to. Unfortunately that means even if the "tutorial" was human-generated, it's probably getting the axe too, unless it has a very interesting or unusual approach, but it isn't clear to me that that is actually a large problem.
Why isn't it? As far as I can remember, all the forums had lot of outright spam, even forums which are orders of magnitude less popular than HN. You can even see spam in usenet. Reddit/Discord is full of it. I wonder are dang/moderators the only reason for this?
I've seen a few comments on HN that looked likely to have been generated by LLMs, but they tend not to get any replies or up-votes so there's not much incentive for people to keep on trying to post them.
---
Please refine: To be honest, I do use GenAI to help me format my comments including this one. I do, however put in substantial input outlining my thought in detail, so I use it more like a glorified grammar checker. It helps me a lot as I'm not a good writer (as being non-native, just sufficient to be able to thrive in business environment, just not a good writer. It helps me tremendously to get my thought across efficiently.
Writing is not just about syntax, but also about communicating who you are. This is your true voice.
Some roughness in my recent writing comes from being on mobile, which makes reviewing even harder! My writing has improved, though, even when I’m working without AI, likely from exposure to better writing flows. I’d say I used to rely more on edits (20% input to 80% edits), but now it’s closer to 75% input and 25% edits. Hopefully, I can refine it even further.
"Consider the following message. Distill the main thrust of the message, and list any implied statements made within. Be particularly sensitive to any negative connotations, and summarize the writer's feelings towards the reader:"
In the case of blatantly false images like on Facebook, they get a surprising amount of organic engagement: https://x.com/FacebookAIslop/status/1806416249259258189
Because it's already filled with russian bots.
Sexual content
Violent or repulsive content
Hateful or abusive content
Harassment or bullying
Harmful or dangerous acts
Misinformation
Child abuse
Promotes terrorism
Spam or misleading
Legal issue
Captions issue
I have noticed that someone immediately calls out pretty much every bit of content as AI generated, whether it obviously is or not. I guess this is a best case scenario.
I do wonder what will spring up next to allow high quality content to be found.
[0] https://wakatime.com/blog/67-bots-so-many-bots
I really don't see how these people making the ebooks are making any money on these books either since they cost 99 cents and they pay for advertising. Maybe they're using llama2 idk lmao, either way I've seen a ton of embarrassing titles pop up on there and don't want people thinking I'm interested in that slop.
We're between a rock and a hard place with the internet's collective signal to noise ratio plummeting and no easy way to fix it.
SEO was annoying but there was still value in the results. Now, as soon as I detect or start to believe some result is AI bullshit I immediately close the tab out.
You know which travel advice I prefer? The one from a small, local blogger that loves their city. You know where I find my music? At house parties. More and more, I also crawl the profiles of individual users for content that I might like.
The cosy web is my refuge from scale and its consequences.
And that lack of scale means it has little mass monetization potential. It's actively hostile to weirdos just trying to make a buck. I don't get randos from who-knows-where trying to hawk their wares in my private group chats, so those recommendations actually carry weight.
just absolutely embarrassing for us all.
We missed the boat long ago on dismantling or neutering the ad industry before it did massive irreparable damage. Whatever we do to fix this mess is still going to leave prominent scars for generations.
It was always possible to cocoon yourself into news and opinion you agreed with.
AI slop does seem a little different, if only because it's machine generated and the sheer amount of it would not have been possible when rubbish was human generated and the gatekeepers of publishing were newspaper editors.
https://theglobalistperspective.substack.com/p/the-rise-of-i...
It seems like the leaders of all of these online platforms are going to just bury their heads in the sand and pretend that this massive problem isn't one.
This AI trend is absolutely going to destroy many (most?) of these companies. For better or worse, the new age of the internet is already here.
I'm not sure of the exact machinations, but the times that my posts have been boosted, it's always been by a publication on Medium where a human editor has read my content and 1) asked that I submit to their publication and 2) submitted to Medium for boosting.
Seems like this type of filter -- as long as they actually keep humans in the loop -- can potentially work to keep high quality content more visible.
Of course, even mentioning the f-word is forbidden, so...
I have noticed a lot of these articles that just make no sense at all starting to appear in my weekly email to the point that I probably won't renew next year if it stays this way, so it isn't just "nobody reading them".
* humans creating things
* algorithms emulating (humans creating things)
* human-in-the-loop guiding (algorithms emulating (humans creating things))
* ai that mimics the (human-in-the-loop guiding (algorithms emulating (humans creating things)))
* what next???
Amongst the others things that walled gardens like Facebook, Instagram, Twitter, and TikTok have taken is the open availability of social signals to rank content. Previously Google used links to build some trust metric, but almost no real person creates public links anymore.
Like for example a TikTok, but all the videos are generated by the platform itself using likes/skips to drive its algorithm to generate more addicting content. That seems like the logical conclusion of where all of this is going.
They can then drop the creator side of the market and a lot of moderation issues that come with it, while gaining full ownership, control, and monetization of the underlying content.
The CEO accused me of trying to extort him because I sent a short email with our findings prior to the publication of this article.
He also casts doubt on the efficacy of AI detection (fair) but I think our AI detection model is orders of magnitude more accurate than the others and I stand by its predictions.
I noticed something, you achieved near 100% accuracy on most domains in every domain but scientific which made me wonder, how much is that could be due to how "strict" and "profrssional" these papers could be, or maybe how a slightly disproportionate number of the training data for these LLM could be from science based articles and papers, as they are generally viewed to be "high quality"
Interesting read either way, best of luck on your project (:
More specifically, humans who feel the need to make a quick buck.
Actually, the problem isn’t those humans, it’s the societal structure creating incentives and pressures that lead to a large percentage of humans feeling like they need to make a quick buck. Whether that quick buck be for cost of living or for social status gain.
> The 12-year-old publishing platform
basically