0.1% synthetic data is enough to degrade AI models (Nature, 2024) (medium.com)

Survey of 65+ papers on model collapse. Key finding from Dohmatob et al. (ICLR 2025): even 0.1% synthetic contamination in training data causes measurable degradation.

No major dataset (FineWeb, RedPajama, C4) currently filters for AI-generated content.

How about complete crap data? We know there are people generating rubbish specifically to feed to "AI". Can they generate enough to cause problems?