Even more important, subjectively, is that this model will run very well on Strix Halo (e.g. Framework Desktop), DGX Spark kinds of devices.
Looking forward to Unsloth dynamic mtp quants.
P.S. Looking at the HF release they already offer Q4_K_M and DFlash drafter for speculative decoding!
This is exactly the kind of model that's been needed in the middle. Realistically self-hosted, Good Enough intelligence, MoE so it's fast on limited bandwidth systems like Strix Halo and DGX Spark.
For a while there's been nothing to run on my Strix Halo that's notably better than what I can run on my dual 32GB GPU desktop (Gemma 4 or Qwen 3.6 dense models), but this seems likely to be the step up in size that actually works better than those.
I love this. Is it possible to give a feel of how this stacks up to the good old Opus 4.5 in coding quality? For me that was the turning point where agentic coding in Claude Code etc became usable. Have we hit that threshold?
Whoa whoa whoa, 118b params, 8b active MOE, long context reasoning, open weights - music to my ears. Hadn't heard of this lab before but I am very excited, will definitely try this out tomorrow - this is a real sweet spot I think in terms of model size and performance.
Testing it now. At the very least, competitive with DS4-Flash indeed. On my small (and per Sol's words, _very_ semantically dense) C test codebase, it found things that only gpt-5.2 managed to find back in the day, but also made a stupidly incorrect initial observation that a memfd_create()/mmap was used for IPC (funnily enough - sol missed that as well in its review, until I pointed it out). Re: the claims vs deepseek v4 - both flash and pro are expected to get a "general availability" release very soon (i.e. well-"post-trained"), so things can change in a... well, flash, as per usual in the current environment.
Looks impressive, and this size fits achievable home hardware.
That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.)
A lot of people are testing it, and reporting disappointed results / benchmaxxxing claim. But do not realize that thinking has a issue with the default configuration.
Important - make sure that THINKING is enabled. By default it wasn't although I was passing the flag --default-chat-template-kwargs '{"enable_thinking": true}' in vllm recipe. The generation_config.json file that is included has by default max_new_tokens as 32k which seems to be cutting off thinking altogether so increase it.
At first I was very disappointed with the output I was seeing, but once thinking is enabled, the code quality seems to be MUCH better. More real world testing to be done.
Really impressive signal that this 128B model can beat DeepSeek V4 (1.6T) on most coding benchmarks!
Also, I really like Poolside's habit to compare not only to other top models in its weight class (others don't do it, looking at you Mistral), but also to the very top open-weight models, even much bigger ones like the 2.5T Kimi-K3!
This is fantastic work, really impressive is an understatement. I really hope this sets a new DeepSeek-esque standard and starts another the death knell for companies continuing to cosplay as frontier labs (like Cohere).
initial impressions, great model for coding, probably swapping it out for qwen 27b for a while to long-term test, more sycophantic than any I've run locally myself
Immediate reaction is that it seems to be a bit behind Meta Muse Spark 1.1 performance at approximately the Deepseek v4 Flash price point. That's quite good given Muse Spark benchmarks a lot better than Deepseek v4 Flash (assuming benchmarks mean anything, which they don't).
45 comments
[ 1.8 ms ] story [ 54.2 ms ] threadsimilar performance to deepseek v4, inkling at size of nemotron 3 super (!)
Even more important, subjectively, is that this model will run very well on Strix Halo (e.g. Framework Desktop), DGX Spark kinds of devices. Looking forward to Unsloth dynamic mtp quants.
P.S. Looking at the HF release they already offer Q4_K_M and DFlash drafter for speculative decoding!
For a while there's been nothing to run on my Strix Halo that's notably better than what I can run on my dual 32GB GPU desktop (Gemma 4 or Qwen 3.6 dense models), but this seems likely to be the step up in size that actually works better than those.
Anyways, keep 'em coming.
The pricing here is incredible. This is the first US release that's competitive with DeepSeek V4 Flash. Very excited about this.
https://github.com/mozilla-ai/otari/pull/348
That said, if someone would kindly quantise this down for the 64GB paupers, that would be appreciated. (I know there’s likely degradation, but some people reported good results with a 2 bit version of Qwen 3.5 122B, and this is starting from a higher point. Would be interesting to try, at least.)
Edit: someone in the process of doing so: https://huggingface.co/vcruz305/Laguna-S-2.1-GGUF
Someone has benchmarked a wide range of different quants.
A lot of people are testing it, and reporting disappointed results / benchmaxxxing claim. But do not realize that thinking has a issue with the default configuration.
Important - make sure that THINKING is enabled. By default it wasn't although I was passing the flag --default-chat-template-kwargs '{"enable_thinking": true}' in vllm recipe. The generation_config.json file that is included has by default max_new_tokens as 32k which seems to be cutting off thinking altogether so increase it. At first I was very disappointed with the output I was seeing, but once thinking is enabled, the code quality seems to be MUCH better. More real world testing to be done.
https://www.reddit.com/r/LocalLLaMA/comments/1v2pg99/laguna_...
Also, I really like Poolside's habit to compare not only to other top models in its weight class (others don't do it, looking at you Mistral), but also to the very top open-weight models, even much bigger ones like the 2.5T Kimi-K3!
host: Apple M3 Max, 128 GB model: Laguna-S-2.1, 118B-A8B MoE, Q4_K_M (75 GB), DFlash speculative decoding server: http://127.0.0.1:8000, llama.cpp, ctx 64K, 8-bit KV cache