LLMs. Diffusion models. The single most important factor in ML models is the data, and the reason GPT works so well is because it was trained on practically the entire internet.
> There was a time not so long ago when “Big Data” was the hottest thing since sliced bread. Developers talked about it. Managers wanted it. Presenters made memes about it. Children asked their parents for Hadoop clusters for Christmas (okay, maybe not quite that far).
> Big Data was going to change the way everything worked. We were about to solve every financial, medical, scientific, and social problem known to humankind. All it would take was a great big pile of data and some way to process it all.
Replace "Big Data" with LLMs and this is all still happening.
Also, "Big Data" is still used today for non-ML statistics and catching criminals / government surveillance. And big clouds, SaaS/PaaS/IaaS are as widely used as ever.
2 comments
[ 3.0 ms ] story [ 14.2 ms ] threadSo what happened? Where did it go?"
Adam Bellemare answers what big data is, its evolution, and how it influenced the shift to modern cloud services and data streaming at scale.
> There was a time not so long ago when “Big Data” was the hottest thing since sliced bread. Developers talked about it. Managers wanted it. Presenters made memes about it. Children asked their parents for Hadoop clusters for Christmas (okay, maybe not quite that far).
> Big Data was going to change the way everything worked. We were about to solve every financial, medical, scientific, and social problem known to humankind. All it would take was a great big pile of data and some way to process it all.
Replace "Big Data" with LLMs and this is all still happening.
Also, "Big Data" is still used today for non-ML statistics and catching criminals / government surveillance. And big clouds, SaaS/PaaS/IaaS are as widely used as ever.