Testing this hardware LLM (LLAMA 3.1 8B on a chip) I get ~16k tokens per second.
With frontier models plateauing, I’ve been convinced AI will end up like bitcoin mining, and that NVIDIA’s general-purpose GPUs will be replaced by model-specific chips.
1 comment
[ 2.4 ms ] story [ 15.1 ms ] threadWith frontier models plateauing, I’ve been convinced AI will end up like bitcoin mining, and that NVIDIA’s general-purpose GPUs will be replaced by model-specific chips.
Glad to see someone innovating in this space.