Common Corpus: the largest public domain dataset for training LLMs

internetter 2y ago ↗

Obviously, there is a lot less data here than other LLMs, but this data might also be higher quality. I'm curious how it works out.

mdaniel 2y ago ↗

uh-huh; if this is any indication, I gravely doubt that: https://huggingface.co/datasets/PleIAs/US-PD-Newspapers/view...

I couldn't figure out their search syntax but both date>2000-01-01 and just 2000-01-01 didn't cough up anything, so I also gravely suspect this data will only be antiquated language usage, too, to say nothing of the objectively horrific OCR job they did

Common Corpus: the largest public domain dataset for training LLMs (huggingface.co)

2 comments