174 comments

[ 1533 ms ] story [ 1504 ms ] thread
the thing is: if AI-Generated Content (if correct, obviously) is just some topics learnt from humans, what's the difference between something correct written by a machine and something correct written by a human?
You can always read Grokipedia if this feels like a lost opportunity to you.
Not sure if you're being snarky or not, but the period at the end makes me think yes
"if correct, obviously" - please elaborate. I do not see how this is obvious. Or how being correct is obvious.

Not to mention this exact question is being discussed in the article.

What about:

Correct information written by AI vs incorrect, biased information written by humans?

Not sure what question you're supposing he asked which was in the article. OP's point seemed to be that correct words written by a human could be effectively the same as correct words written by a machine, supposing the machine managed to be correct by learning from human content. The 'obvious' part is this wouldn't be true if the AI content was incorrect. At least that is what I got.
If restaurant asks to only eat the food you bought there at their tables, the question is not "but what if food from other places also is good"?

Can't wikipedia ask who and it what way writes content on their pages?

There will be a point in the future, possibly near future, when AI will be capable of reliably producing encyclopedic content, and we'll have tools to verify that the content produced passes reasonable standards.

Today is not the day. Especially not regarding the verification.

That's really not how probabilistic systems work though. They're great at getting 80% of the way to the goal but the last 20% is always excruciatingly difficult. I really don't intend that a knock because LLMs are useful, but they also aren't magic boxes where anything is possible.
This is the problem that a lot of user-content based sites, and open software projects, are experiencing. The tools make prodiving 80% correct submissions so easy that many people are doing that and thinking that it is fine not to bother verifying for correctness (or not even knowing how to verify correctness). This means those projects are being inundated by submissions that are 20% tosh or worse.
Another problem with user content sites, these numbers are completely pulled from the ass and are a meme that just gets repeated ad nauseam. They mean nothing, but there will be people that take it seriously.

Is 80/20 ever generally correct? Why is it favored over the 90/10 meme? Why would it apply to this? Why wouldn’t the numbers be vastly different for different domains of content generation? Etc.

If you pay attention, so much user content is just repetition of something they read on the same fucking website and uncritically checked into their brain.

Current HN top post is Roblox downloaded 145GB models from us (including adult content) without asking. I fully expect to start seeing comments to the tune of “Roblox is stealing porno to train AI for children” within the day in any discussion Roblox comes up in.

Why would these folk’s contribution to any other user content site be any more thoughtful?

80/20 doesn’t mean literally 80% and 20%

It’s just a rough way to say “mostly this with a decent bit of that” because 20% seems like a medium-small percent that could or could not matter depending on the context

That’s all anyone means when they say 80/20

To your second point, the 80% could be lots of basic things and the 20% could be the important facts, so it’s not always meant as a dismissal that the 20 doesn’t matter

> Is 80/20 ever generally correct?

Sometimes. But it, or 90/10, or sometimes 75/25, is understood to main “a large part of the time”/“a significantly smaller but still crucial amount off the time”.

> If you pay attention, so much user content is just repetition of something they read on the same fucking website and uncritically checked into their brain.

Which is exactly the problem. We now have LLMs doing the same thing, flooding everywhere with more of that noise, further hiding what good signal remains. Even if some of that good signal is by chance almost entirely LLM generated too, it appears to be at best a vanishingly small amount.

Grokipedia serves as an existence proof that it is possible.
Funny, I wanted to mention grokipedia as an example of why a pile of slop that’s subtly wrong about many details is a bad idea.
So is Wikipedia, but with grokipedia you can at least correct it even if there is not a journalist pointing out the obvious.
Does Grokipedia even accept contributions that don't align with Musk's political bias?
You are free to give any suggestion that you want.
I don't think anyone's arguing against whether or not it's possible, the question is whether it's valuable, correct, and A Good Thing.

I have access to a number of LLMs and can order it to build me a wiki site with all knowledge of the world, just like everyone else with the same access. What would make mine any more valuable or better than another?

Grokipedia's mistake is that it generates text statically instead of dynamically from a machine curated data structure in latent space.

Latentpedia starts from wikidata, not Wikipedia.

Grokipedia lacks the fundamental quality that makes WP great, collaboration.
Saying there is no collaboration is like saying multiple people on a software project can't collaborate with Claude Code because it's really just Claude writing the code.
Even ignoring accuracy, this ignores the issue of data bias when training such models.

Wikipedia already struggles with issues in idealogical bias from authors - but thankfully human checks help (somewhat) to mitigate that.

Even subtle bias can manifest in significant ways. Given that the predominant market leaders in producing such models are currently : A country that's attempting to squeeze the word "america" in any geographic formation possible + rewriting history to put itself first, and a country that has perfected censorship en-mass and already actively doctors its own history.

I'm not convinced giving LLMs carte blanche to write authoritative sources that are already prone to isn't just needlessly reckless and short-sighted.

Coding and math are easily verifiable, language and prose is not.

As a result, I don't think LLMs will reach this state easily.

Latentpedia.org will not have these problems. The content is human produced, but the extraction of a graph and creating the latent space is AI driven.
Almost all of these websites or platforms are doing these massive cleanup to remove ai content from them. Well appreciated.
[delayed]
Wikipedia is also a training source and shouldn’t also be a model output for content. Eventually it will be an oroborus, but it seems prudent to avoid that for as long as possible.
The models are already trained on model output on purpose
Maybe so, but in a controlled and deliberate fashion.

Not just random output of who knows what model, from who knows what prompt.

A little over a year ago, the MIT Technology Review ran a piece about how outputting AI content to non-English Wikipedia is rapidly corrupting the primary training data for uncommon languages because of exactly this process: https://www.technologyreview.com/2025/09/25/1124005/ai-wikip...
[delayed]
It certainly may place a very high value on archive.org over time thanks to their historical timestamping
Ooh, I like this way of thinking of it. Thanks for that!
Hence all the AI labs buying up and destroying physical books. Even "yellow press" stuff, pulp fiction, dime novels and whatnot "low quality" literature are a valuable source of how actual humans write.

I wouldn't even have much of a problem with it - if these labs would be fair enough to humanity as a whole to provide the raw scans after the process is completed, and if they would limit themselves to books that have been on the market for more than 20 years.

And destroying!?
From my understanding, they are legally obligated to in this case.
The Holy Inquisition was also required to burn witches in the medieval era.

That does not really make it a law though.

What do you think bookstores do?

https://en.wikipedia.org/wiki/Stripped_book

> A stripped book is a mass market paperback that has been stripped of its cover in order to be recycled. The covers are returned to the publisher as evidence that the books have been destroyed and to obtain a credit on the purchase price.[1] The books are meant to be destroyed by pulping but some find their way to street merchants, sometimes by illegal means, and are resold.[2]

I grew up reading books with this ditty in them:

> If you purchased this book without a cover you should be aware that this book is stolen property. It was reported as “unsold and destroyed” to the publisher and neither the author nor the publisher has received any payment for this “stripped book.”

Those books certainly weren't scanned before being pulped. Would've been illegal. Really, the big thing that destroys books is the copyright term being as long as it is.

AI companies weren't buying up tons of unsold mass-market materials, and I think it's disingenuous to even make the comparison.

What was going on recently was AI companies were specifically seeking out rare — or sometimes irreplaceable — books to supplement their data sets (and hopefully getting a leg up on their competitors). The destruction of the books wasn't the point, it was just a consequence of the business-oriented realization that they could scan the contents of more books faster if they just chopped off the bindings rather than investing the time to care for the books and keep them intact (plus supposedly playing along with copyright law by "only having one copy of the copyrighted material").

Every tor SF book I've ever read had that. Neat little bit of history
> if these labs would be fair enough to humanity as a whole to provide the raw scans after the process is completed

Unfortunately, this would get them an even bigger copyright smackdown than they've already received.

Wait, they received a copyright smackdown? When?
A human did this first with Scots Wikipedia by rewriting English articles in Scottish eye dialect of English and passing it off as Scots.
AI is great at making poor writing good. I've had good luck having AI turn my rough draft into something readable. (just check my comment history to see how bad my rough drafts normally are). AI is also reasonably good at research and fact checking. But in all cases you need to think before you accept anything AI says. AI generated content all too often hasn't been checked, some is good but some is awful and an expert needs to decide which is which.

That is AI is a tool. I've driven screws with a hammer - it worked in a pinch, but it was not good. People have also used hammers to kill others.

> I've driven screws with a hammer

That takes a lot of commitment, you really have to want that screw in pretty badly! And when you're done it ends up just being a worse nail.

There are nails that have threads on them that are made especially for this purpose.
No, there aren't. Ring shanks have ridges, not threads.
I think they were talking about spiral shank nails, not ring shank. It has a twisted grove along the shank of the nail, sort of like an extremely course thread.

I've seen these used to assemble wooden palettes (they are VERY hard to remove). A search on Home Depot reveals they may be used for certain types of siding as well.

I built a house with them. You really want to check twice if you're putting them in where you want them because getting them out is - as you noted - annoyingly difficult. But they're very nice for large surfaces that are going be battered by the wind, they're much longer than your typical screw and working with them is very fast.
Ring shanks do exist, but they are also made with a softer iron so it can be struck with a hammer and bend instead of fracture or snap off.
It makes poor writing readable, not good.
It makes poor writing look superficially more considered. More readable, no, not when you realise.
It seems good. And that's a major problem, one of the signals for quality is not useless.
I'd much rather see your rough drafts than whatever AI turns it into.
Depends on if they are verified or just accepted.
Unsubstantiated claims is one thing.

If it doesn’t act to fix the politically Left biased perennial sources list, it will continue to be a propaganda machine.

> it becomes noisy and sameish at larger scale or the more you read it.

I absolutely understand where you are coming from, and I agree to a point. If you're of an age where you actually read a printed encyclopedia, the tone and voice of the entire set was sameish as well. The difference is that it didn't sound LLMish like LLM text does. I know that printed encyclopedias would have been vetted by editors, and I feel this is the key difference. Editors not only knew grammar rules, but had that sense of good writing practices that's hard to write down as rules. A piece of text passed through a good editorial review will read human while LLM text reads as LLM text.

what's funny of that the llm learns from Wikipedia and then is used to put data back in Wikipedia. can't imagine that going well lol
I think this is exactly what we should do. Send agents to collect evidence and write deep research reports, then train on those reports. DR reports are derived from hundreds of sources and can map out the distribution, what are the opinions on a topic, not necessarily what is the one correct answer. Agents could write one piece about every human, place, topic, event or entity there is under the sun. They could also try to generate the same without sources (closed-book mode) and report on LLM biases on that question.
Please let's not clutter up wikipedia with new articles for every entity in the universe. It would be too much for humans to review. Though it sure would be funny to see an AI-peida as you describe.
What you're describing though is that individual source of content tends to sound like itself. That isn't a problem so much as what we tend to want and expect.

When we say AI sounds same-ish we mean all sources of content can sound the same - fiction and non-fiction, news articles and blog posts, marketing pitches and ai drafted personal messages, all sound kinda the same. That is a different situation, and that's before you get to "and that one sound is LLMish".

A lot of AI content is just wrong - hallucinated lies. Just look at Grok-slop by the right-arm raising billionaire clown. It looks neat on the surface but once you dig in you begin to see more and more errors, thus wasting your time.
100%. Looks like they're using pangram (hope they're getting that for free...) which also flags stuff like Grammarly. That one's an interesting gray zone to me that I've not thought off before. LLMs whales doing the heavy lifting is one thing, but small edit tools like grammarly that "improve" your phrasing are likely a massive algea like contributor to what is out there too.
Why for free? Wikimedia is very well funded, and I'm sure that Pangram could use a leg up...
They're providing the training data for free too. I'd think there's mutual interest there.
what ! according to pages longs of ads if I don't donate all my money to Jimmy Wales Wikipedia dies!
I've felt that like tools like Grammarly are harmful for native speakers. I get if it's not your first language you need extra help, but if you are a native speaker these tools only serve to homogenize your way of speaking.

Having a program constantly suggest "better" ways of phrasing things is stripping your personality from your language. Maybe this is needed if you are writing corporate PR pieces, but you should avoid it for anything else.

They are probably doing at the request, and in the future even paid for, by LLM providers...
As the biggest cheerleader of AI, I like to have a few sources that are free of AI and Wikipedia is one of the few perfect places to embrace this.

Of course AI can be used to check sources, get critical feedback and translate to languages. But content should be 100% human driven.

Much of wikipedia is written by one person, Steven Pruitt. He has created more than 33,000 Wikipedia articles. People rightly credit him for cataloging obscure topics even though much of what he writes is rote.

But now when AI does it, it's deemed bad? If it's factual and at all useful to someone, why not leave it? I feel like we're waging the same battles when Wikipedia became an open encyclopedia. Surely you need to control who can edit right? We all know how that ended.

https://en.wikipedia.org/wiki/Steven_Pruitt

(comment deleted)
Note that Pruitt has the highest number of edits on Wikipedia, but not the highest number of created articles.
The problem with AI on Wikipedia is that the reason I've noticed it is because I've just seen blatantly incorrect stuff, mentioned breezily in passing, with no references. It's shitting up a thing that was already pretty easy to shit up if you had a motive, and your changes often wouldn't be noticed for months or years if an article wasn't well-trafficked.

AI doesn't even have a motive, it's just dumb - so you can't even track the mistakes to specific motivated users or networks of users. There's no way to clean it up.

Although I have to agree with you (probably, because I'm going to put words in your mouth) that this is a basic problem with wikipedia and how it is structured and governed, rather than a problem with AI. Anybody could always just put nonsense anywhere at any time; AI is just a chaos monkey. Wikipedia can't rely on groupthink to maintain itself, it has not worked and will not work, chaos monkeys are needed.

Ironically enough I just started an "Encylopedia of AI" project, and I've decided to generate all of the content with DeepSeek, with me being an editor/reviewer. I'm not really doing it for anyone except myself but I'm quite enjoying it so far. https://enkuklios.com/
If we leave out environmental (energy, datacenters, coolant) and political arguments (copyright stealing, private data harvesting etc.) even if we agree that those are "bad".

What then, is the problem of AI-Generated Content in Wikipedia?

> Fake or misleading citations have become a major worry.

Why is this not a standalone problem? Humans can fake and mislead in citations too, in fact, this is a major problem and has been so, for literally centuries in science.

> Being a place that was, in theory, free from AI slop has become part of Wikipedia’s identity.

Isn't this a separate issue in itself too? One of "quality". If AI generates poor quality content, than I would think the poor quality is the problem, not the producer of it.

The article mentions several times how hard it is to detect AI content. If it is that hard, then obviously AI is doing a good job and producing content that matches that of humans. Why then insist on one type of producer?

Poor quality is the problem and since the bandwidth of editors doesn't magically grow with the amount of input, rejecting AI content is a very effective way to keep average quality up.
>Isn't this a separate issue in itself too? One of "quality". If AI generates poor quality content, than I would think the poor quality is the problem, not the producer of it.

Yes, but who will review the mountains of slop to determine whether it is of acceptable quality? The main problem AI introduces is high volume at low quality, often overwhelming the human resource bottlenecks these systems have in place.

Makes sense indeed, but then the quality nor the volume is the underlying problem, but the review process.

Apparently Wikipedia has good reasons to not change this process fundamentally, other than add extra filters at the start to try and detect low quality posts, based on attributes that show a correlation with low quality.

Sounds like a pragmatic, but far from ideal solution to me. Correlation is not causation.

About a decade ago, I read the history of AIDS on Wikipedia. It was a very informative (and satisfying) 20 minute read.

Now it's pages and pages and pages of text: probably factual, but not a satisfying read. It feels like it's for a different audience, and it's not satisfying to read.

Is it AI generated? I don't know for sure, but it feels like either someone (or something)'s missing the point of who Wikipedia's audience is; or that someone is following an agenda, (or that there's an edit war going on between dueling agendas and the facts are added to drown out the "AIDS is a consequence of bad morals" agenda.)

Same feeling here - not sure if it’s AI, or zealous wikipedians + incentives that make addition easy but editing and pruning means stepping on many other writers turf, but I used to enjoy reading articles, while now many are more useful as “information fodder” for LLM to summarize and interpret.
I don't think it has anything to do with AI. Wikipedia has been like that for at least a decade for me.
Just so we're on the same page, this is the one I just skimmed:

https://en.wikipedia.org/wiki/History_of_HIV/AIDS

On reflex, I searched for "reagan". 0 results as of the morning of October 27, 2026.

I was there in the 1980s when the AIDS outbreak started getting mainstream news coverage. I watched the Ryan White movie on TV, when it was the biggest topic at school for months afterwards:

https://en.wikipedia.org/wiki/Ryan_White

I can tell you what really happened. HIV/AIDS scared the bejesus out of the political establishment of the time. Who then stoked that fear to push their socially conservative agenda onto everyone in an attempt to return the US to what they viewed as a morally just culture (basically the 1950s), much like MAGA today.

And just like when Donald Trump refused to wear a mask to set an example in protecting the elderly and other vulnerable groups from infection by COVID-19, Ronald Reagan refused to devote real resources to slowing the spread of of AIDS and finding a cure. Because AIDS furthered the GOP's efforts to expand the War on Drugs, push abstinence-only sex education, demonize same-sex relationships and criticize secular culture in general.

In a very real sense, Ronald Reagan and his administration let AIDS grow from being a manageable disease to becoming an epidemic that killed millions of people. His misplaced priorities set back research and treatment options by decades.

Omitting Reagan's name from the history of AIDS is like casually leaving Benedict Arnold out of the history of the American Revolutionary War.

The domino effect of his abdication of duty has touched everything from US police interventions in developing nations that it's exploited to its $40 trillion national debt. It's literally fogged the lens through which we see the world.

Its easy to forget all of that now that once-daily HIV treatment pills and bi-monthly preventative injections exist, and we're finally seeing cures on the horizon.

Revisionist history now dominates our political discourse. I write this because the "AIDS is a consequence of bad morals" agenda is not something to be lightly swept under the rug. It's literally a pillar upon which the alternate timeline we're on now stands.

I was living and working in Kokomo, Indiana when the Ryan White situation occurred. There were other kids who got AIDS/HIV from hemophilia meds like Ryan. My coworkers were thrilled that several of these kid's homes were burned to the ground by people who didn't want those kids going to "our" school. Those coworkers were also proud that one of the Grand Wizards of the Klan lived in Kokomo.

I remember Reagan manufacturing AIDS into some sort of plague sent by God to punish brown skinned alongside gay people. I'm not surprised that historians have whitewashed Reagan's racism.

It was Reagan's administration that got the Fairness Doctrine repealed. That single action made the right wing media possible. Fox could never have existed prior.

The HPV vaccine prevents several cancers. Australia, which has mandatory HPV vaccination, is seeing a 95% drop in the cervical cancers that the vaccine prevents compared with pre-vaccine people. In my state, as well as some other red states, want the HPV shot banned or at the very minimum requiring positive parental consent.

also the quotations or randoms instead of concise text - feels like something written by 5th graders
I don't follow Wikipedia policies, but I imagine the solution is to break it up into multiple pages, and replace each long section (e.g. Origins) with a 1-2 paragraph summary.

And while I agree it's too long, do note that about 50% is the References section.

> It was around December 2023. A few other editors and I were chatting on Wikipedia’s Discord server, and we had all seen very strange articles with the same odd characteristics.

I was afraid of that. I grabbed a 2024-04-20 copy as a backup, in case it turned to slop, but I suspected it might be tainted already.

Good luck with that. I can’t see how it’s even possible, today, let alone in the future.
I wonder if the AI companies would even pay for this, because it increases the quality of the training data.
That would be good, as I imagine that wikipedia is in survival mode these days, as less and less people visit their site directly Even so more people indirectly benefit from its content
It's not in survival mode, they have a pretty large endowment and low dev spend they'd like to increase. The number of edits per week has been roughly steady for years, and their revenue per year has been increasing steadily. Their hosting costs are hyper-optimized and tiny; they spend much more on programmers and lawyers.
On the same note, I wonder why AI almost never gives me links to Wikipedia. Do they have it in training data without any attribution?

It feels like the distribution of sources/links it outputs is 33% Reddit, 33% Random blog, 33% Not Found.

The LLM weights are not tuned for truth, including citing sources, it's tuned for "doing as much as possible, as fast as possible, on your own". If you ask it for a source, it literally has to autocomplete one, it's not going to figure out which source is the most appropriate or authoritative as part of that text generation pass.
> I wonder why AI almost never gives me links to Wikipedia

Users on Reddit say…

I attempted to fix some AI-generated content that I found on wikipedia recently - it had broken formatting, confusing + made-up text, and links on the wrong keywords; just generally very low quality edits.

I made my edits and they were immediately reverted by wikipedia's own AI bot. I gave up.

Which page? Let me see if I can help.
You won’t get a usable reference. Approximately nobody who complains about Wikipedia ever gives links, or even article titles.
I'm not trying to dox myself
Links or it didn’t happen.
What, and invite an off-site brigade?

The only way to win is not to play.

You're allowed to revert ClueBot if it makes a mistake. It won't revert your revert, although it will flag your revert for human review.
I mean this is good, because AI generated content is explicitly against the guidelines, however I can't help but think there is some utility to be had with AI and online encyclopedias, and we might be missing out on that.

Of course I'd love for this experiment to be done outside of the main site, and only integrated when it's mature, stable, and proven to be actually useful, but other than that implementation concern, I feel this is something that needs to be explored.

You might find grokipedia and latentpedia interesting as two such attempts.
Is it great to have both. The AI-pedias will probably contain a lot of hallucinations, but also the information that gets removed from Wikipedia (by deletionists, or by editors with an agenda). Neither is reliable, but it probably makes sense to check both.
(comment deleted)
Wikipedia can gain a lot by doing a cleanup using AI instead. There’s a lot of incorrect information in Wikipedia that you can easily detect using AI with a prompt like “find something wrong in a few randomly selected Wikipedia articles”. Usually it does find something.

If it’s that easy for me to correct it, it should be automated.

They already have that. Look around in this thread.
Its much appreciated. Reminded me of runescape (a game) wiki devs blogpost, its an interesting read: https://oldschool.runescape.wiki/w/User:Cook_Me_Plox/OSRS_Wi...
> The median new plugin used to end up with a couple thousand installs. It's now 28.

This feels like a microcosm of software development as a whole.

There's more software than ever, yet there are fewer people engaging with said software than ever, and the good software gets buried under a giant pile of crap.

There is more code than ever. The amount of software seems to be shrinking.
that amount of new plugin development is crazy when using unauthorized plugins can get you banned from the game
I’ll never understand this desire from certain OSRS folk.

You can put hundreds of hours into these accounts, and people will roll the dice on a vibecoded plugin they (or a stranger on the runelite plugin repo) make with very little understanding that Jagex (the company that maintains OSRS) just sees client commands. If you look like a bot and smell like a bot, you get treated like a bot.

I now know two friends that got bans (one permanent!) for rolling their own unpublished plugins, not realizing that they asked Mod Claude for a macro set for some bossing content… oof.

Poor guys. I'll never understand why it's so attractive to generate tons of code that you clearly don't understand, and then especially to go on and push it onto humans who do strive to understand.
“Hey Claude / Codex / Gemini / Deepseek / Whatever, look over this pile of steaming LLM crap and write me a paragraph on why I should reject it. If it’s good and you don’t tell me I’ll be mad. Make no mistakes”
In both cases this is caused by poorly (human-) designed software, that assumes you go through a similar development cycle to the author, would that be part of the spec? of course not. This is just the obvious end result of an industry and culture not taking formal correctness seriously well... ever.

And so the goalposts move, oh now we've always cared about user resources, AI is using too much, but in 3 months when it's better than all human programs at resource management what will they move onto? Did we forget that unused resources are wasted resources? Or can that only justify poorly human-written programs.

Effectively this is a project to help evolve AI to pass the Turing test for Wikipedia contributors.
Some high quality RLHF from the wikipedia editors for free.
All of these companies helped themselves to resources like Wikipedia and then turn around and allow their tools to dump slop everywhere. It's appalling.
Wow what a nuanced take from Wikipedia. Despite taking such strong efforts to protect human input, still admitting it has a place on the platform:

> Yesterday, AI was something to clean up. Today, it’s a tool to help us with the cleanup. Tomorrow, it will be the air we breathe.