In the early days of ML research they started building expert systems. One of the things that came out of that was quality data mattered more than anything else. Where the hell is anyone getting enough quality data to feed an LLM now? Were building language models with shit content, and garbage in, garbage out...
I was particularly baffled by google's deal with reddit. they've spent 60 million (I think) to plug the sewage outlet of the internet directly into their training data
4 comments
[ 4.9 ms ] story [ 22.8 ms ] threadThere ought to be a term for that quality achievement level..
Two Sigma? :)
In the early days of ML research they started building expert systems. One of the things that came out of that was quality data mattered more than anything else. Where the hell is anyone getting enough quality data to feed an LLM now? Were building language models with shit content, and garbage in, garbage out...