Pretty amazing for being trained from scratch, to have ~Qwen 3.8 27b quality with such an increase in tok/s performance. Excited to try it out and see how snappy it is.
I really like Muse Glimmer, and on low reasoning level it's both helpful and quite quick.
But I won't turn down a model that is faster.
Nemotron Bolt appears to be really quite good but unfortunately it is quite slow on macOS/Metal in llama.cpp at the moment because of lack of support for a tensor type (or something… not sure I understand).
7 comments
[ 0.32 ms ] story [ 13.5 ms ] thread- 125 tokens per second on an M4 Pro MacBook Pro
- 512k tokens context window
- 40B total, 2.5B active parameters mixture-of-experts
Pretty cool release!
I really like Muse Glimmer, and on low reasoning level it's both helpful and quite quick.
But I won't turn down a model that is faster.
Nemotron Bolt appears to be really quite good but unfortunately it is quite slow on macOS/Metal in llama.cpp at the moment because of lack of support for a tensor type (or something… not sure I understand).