Ask HN: Isn't most of the magic of AI the embeddings?

7 points by cushpush ↗ HN
Context embeddings describing language are the secret sauce, are they not?

3 comments

[ 3.8 ms ] story [ 20.7 ms ] thread
I feel the need to write yes 50 times.
That’s like saying the magic of a model is its predictions. The magic is really the process of gathering data, cleaning it, creating a model architecture, and meaningful objective function for the model that creates the embeddings.

Embeddings themselves aren’t new. It’s how they’re being trained - and the incredible scale - that’s new.