Hi folks!
I love running local small models, but it's quite annoying to attempt to compare them using benchmarks. Not only are they scattered around, they often don't even share the same ones.
So I built Tiny League for those of use who care about small models. By small, I'm aiming at 3-200B, which is a range that I consider both capable and able to run at home (for those who have unified memory like mac, strix or spark)
I'm limiting the model list to models released after April and excluding models that are trained for specific use-cases.
1. works fine for me, are you sure you don't have any other filters active that might result in 0 models?
2. good idea, a few people have requested that. It becomes a little bit more difficult when models have engrams, but I will consider!
3. There are some tools to do that, I personally don't like them. I understand the convenience but I am staying away from that. There are many factors at play, not all models work the same way, even if they use the same VRAM. Not to speak of using quants and how each quant may affect a model differently
5 comments
[ 0.23 ms ] story [ 15.7 ms ] threadSo I built Tiny League for those of use who care about small models. By small, I'm aiming at 3-200B, which is a range that I consider both capable and able to run at home (for those who have unified memory like mac, strix or spark)
I'm limiting the model list to models released after April and excluding models that are trained for specific use-cases.
Appreciate all sorts of feedback :)
1. works fine for me, are you sure you don't have any other filters active that might result in 0 models?
2. good idea, a few people have requested that. It becomes a little bit more difficult when models have engrams, but I will consider!
3. There are some tools to do that, I personally don't like them. I understand the convenience but I am staying away from that. There are many factors at play, not all models work the same way, even if they use the same VRAM. Not to speak of using quants and how each quant may affect a model differently
This is what I see when I check the MoE checkbox. The list at the top of the page has no models: https://paste.dbohdan.com/1nagxp2-s90bk/screenshot.png.