How is this easier than using lmstudio or omlx or whatever your favourite runtime is ?
It’s maybe a bit interesting that Jetbrains are making moves to integrate with local models more closely, but I think claiming this is “easier” is incorrect for most users.
I get up to 90 tokens / second with Jundot/Qwen3.6-35B-A3B-oQ6-mtp, on a M3 Max with 64GB. MoE so it is not a dense model, but that means it runs faster (also, mtp helps). It is a 30GB model, but you should be able to load it, otherwise try the 4-bit quant instead of the 6-bit quant, don't bother quanting your KV Cache (don't enable turboquant in oMLX), since that will slow you down.
As far as I know, if you want MTP you need to serve it with either MTPLX or oMLX. oMLX is more stable and user friendly.
I’m using Goose but have tested Codex and OpenClaw among others. Codex worked fine but I couldn’t get web search to work with Qwen. OpenClaw I could get web search to work but its system prompt eats a lot of context tokens. Goose works for web search if you use SearchXNG, and has fairly slim system prompts. I’m still evaluating DeepSeek Harness and might move to that once it’s more stable. I tried other more obscure options but they had some flaws that made me settle on goose.
Why are they tying the client with the AI model instead of using OpenAI or another popular http format? Running llama.cpp locally is about the right level of complexity for most doing local AI.
12 comments
[ 1.8 ms ] story [ 18.8 ms ] threadIt’s maybe a bit interesting that Jetbrains are making moves to integrate with local models more closely, but I think claiming this is “easier” is incorrect for most users.
I’m using Goose but have tested Codex and OpenClaw among others. Codex worked fine but I couldn’t get web search to work with Qwen. OpenClaw I could get web search to work but its system prompt eats a lot of context tokens. Goose works for web search if you use SearchXNG, and has fairly slim system prompts. I’m still evaluating DeepSeek Harness and might move to that once it’s more stable. I tried other more obscure options but they had some flaws that made me settle on goose.