The early exit can be done without an extra flag, but increased code size.
Long term is fuzzy, but the 3D chip tech seems poised to offer exactly that: massive cache, GPU and CPU cores. NVidia isn't going to play nice because they don't like where this positions them in this future. AMD, Intel…
The 1.4T tokens are what the model was trained on, and not the token range of the embedding.
The early exit can be done without an extra flag, but increased code size.
Long term is fuzzy, but the 3D chip tech seems poised to offer exactly that: massive cache, GPU and CPU cores. NVidia isn't going to play nice because they don't like where this positions them in this future. AMD, Intel…
The 1.4T tokens are what the model was trained on, and not the token range of the embedding.