HN Explorer

14 comments

[ 3.2 ms ] story [ 194 ms ] thread
EagnaIonat
Tried out the Ollama version and it's insanely fast with really good results for 1.9GB size. Supposed to have a 1M context window, would be interested where the speed goes then.

No Mamba in the Ollama version though.

mehdibl
Ollama default to Q4 usually and 8/16k context and not the 1M context
serioussecurity
Every technical paper I've read that IBM publish at an ML conference has been P-hacked to hell. Stay away.
aetherspawn
I really just want to know how it compares to ChatGPT and Claude at various tasks, but there aren’t any graphs for that.
KronisLV
It will probably take a few days/week for some in depth benchmarks to start popping up.

The IBM article has this image showing that it's supposed to be a bit ahead of GPT OSS 120B for at least some tasks (horrible URL but oh well): https://www.ibm.com/content/dam/worldwide-content/creative-a...

So in general it's going to be worse than GPT-5 and also Sonnet 4.5, but closer to GPT-5 mini. At least you can run this on prem, but none of the others. Pretty good, could possibly replace Qwen3 for quite a few use cases!

anshumankmr
Also worth checking out was codestral... I think that had a 256k context and used Mamba even if it is slightly older model now... it had worked great for a Text2SQL use case we worked on.
incomingpain
Magistral 2509 just came out. It super slows down when you go over 40,000 context. It's quite a fantastic model.
incomingpain
"Small" is 32b a9b for 19GB @ Q4_K_XL

20GB @ 100,000 context.

But for some reason... LM studio isnt loading it onto gpu for me?

I just updated to 0.3.28 and still wont load onto gpu.

Switching from Vulkan to rocm. It's now working properly?

https://docs.unsloth.ai/new/ibm-granite-4.0

Fantastic work from unsloth folks as usual.

As it's running in roo code, it's using more like 26GB of vram.

~30TPS

Roo code does not work with it.

Kilo code next. It seems to be about 22GB of vram.

Kilo code works great.

The model however didn't 1 shot my first benchmark. That's pretty bad news for this model given magistral 2509 or apriel 15b are better.

Better on pass 2, still no 100%

3rd pass achieved.

Im predicting it'll be around 30% on livecodebench. Probably like 15% on aiderpolyglot. Very disappointed in its coding capability.

I just found:

https://artificialanalysis.ai/models/granite-4-0-h-small

25.1% on livecodebench. Absolutely deserved.

2% terminal bench.

16% on coding index. Completely deserved.

thawab
After getting burned by Watson. I am not touching any AI from IBM.