Primary sources of AI training data

2 points by kishoredbn 3y ago ↗ HN

What are some of the primary sources of public data that are utilized in the training of large AI models?

1 comment

[ 3.3 ms ] story [ 15.6 ms ] thread

The Facebook Llama paper[1] calls out a number of the ones they used.

Briefly, they include:

- CommonCrawl

- Wikipedia

- Github

- StackExchange

- arXiv

and "Books" which appears to be made up from Project Gutenberg books among others.