This isn’t surprising to me at all. These services put too much trust in “the most common answer” when that might not be the correct answer. Just because people think one thing doesn’t make it true. It’s super easy to spread misinformation online. And if you can SEO to the top, the AI will think your site is correct.
I see factually incorrect “ai summaries” in search results all the time and see that it cites ai-generated slop blogposts that SEO-hacked themselves into taking up the entire first page of search results. This is most common for recent stuff where the answer simply isn’t certain but these AI services will assert something random with confidence.
Not even for news stuff specifically, I’ve been searching about a new video game that I’ve been playing and keep getting misleading obviously incorrect information. Detailed, accurate game walkthroughs and wiki pages dont exist yet so the ai will hallucinate anything, and so will the blogspam articles trying to get SEO ad revenue.
I'm a bit suspicious of this report - they don't reveal nearly enough about their methodology for me to evaluate how credible this is.
When it says "The 10 leading AI tools repeated false information on topics in the news more than one third of the time — 35 percent — in August 2025, up from 18 percent in August 2024" - 35% of what?
Their previous 2024 report refused to even distinguish between different tools - mixing the results from Gemini and ChatGPT and Perplexity and suchlike into a single score.
This year they thankfully dropped that policy. But they still talk about "ChatGPT" without clarifying if their results were against GPT-4o or o3 or GPT-5.
AI will create ever more AI-generated synthetic content because current systems still can't determine with 100% certainty whether a piece of content was produced by AI. And AIs will, intentionally or unintentionally, train on synthetic content produced by other AIs.
AI generators don't have a strong incentive to add watermarks to synthetic content. They also don't provide reliable AI-detection tools (or any tools at all) to help others detect content generated by them.
Why does this post not take into account the other side of it? AI has helped me become more grounded and correct in a lot of areas. Intricate questions are now easily answered in ChatGPT.
Not only that - grok use in twitter works surprisingly well. Can some one really quantify the effect it has had in countering fake news?
It is now way harder to spread fake news on X because a simple grok tag can counter it.
12 comments
[ 4.9 ms ] story [ 31.8 ms ] threadI see factually incorrect “ai summaries” in search results all the time and see that it cites ai-generated slop blogposts that SEO-hacked themselves into taking up the entire first page of search results. This is most common for recent stuff where the answer simply isn’t certain but these AI services will assert something random with confidence.
Not even for news stuff specifically, I’ve been searching about a new video game that I’ve been playing and keep getting misleading obviously incorrect information. Detailed, accurate game walkthroughs and wiki pages dont exist yet so the ai will hallucinate anything, and so will the blogspam articles trying to get SEO ad revenue.
examples:
all they will be training on now is spamanyone that says "AI is the worst today it will ever be", no
because that was before the world reacted to it
> one of the most basic tasks: distinguishing facts from falsehoods
I do not think that is a basic task!
I wonder how it compares to the rate of growth of false information in traditional news?
I feel like false information masquarading as "news" on social is rapidly increasing (and that rate is accelerating)
When it says "The 10 leading AI tools repeated false information on topics in the news more than one third of the time — 35 percent — in August 2025, up from 18 percent in August 2024" - 35% of what?
Their previous 2024 report refused to even distinguish between different tools - mixing the results from Gemini and ChatGPT and Perplexity and suchlike into a single score.
This year they thankfully dropped that policy. But they still talk about "ChatGPT" without clarifying if their results were against GPT-4o or o3 or GPT-5.
AI generators don't have a strong incentive to add watermarks to synthetic content. They also don't provide reliable AI-detection tools (or any tools at all) to help others detect content generated by them.
Not only that - grok use in twitter works surprisingly well. Can some one really quantify the effect it has had in countering fake news?
It is now way harder to spread fake news on X because a simple grok tag can counter it.