Author here. I'm comparing local inference options on the same hardware. This is the second post, after Ollama vs llama.cpp; vLLM vs Ollama and Unsloth are next. Suggestions on what to measure are welcome.
Setup: one RTX 3090, Fedora 44, the same Qwen3.5 9B and 27B Q4_K_M GGUF files in both tools, LM Studio's headless daemon.
- Single request: Ollama about 10% faster at 9B, about 5% faster at 27B.
- 4 concurrent requests at 27B: LM Studio 90.2 tok/s total, Ollama 33.1, because Ollama spilled part of the model to the CPU.
- Disk: Ollama copies the GGUF; LM Studio can symlink it.
1 comment
[ 2.5 ms ] story [ 9.7 ms ] threadSetup: one RTX 3090, Fedora 44, the same Qwen3.5 9B and 27B Q4_K_M GGUF files in both tools, LM Studio's headless daemon.
- Single request: Ollama about 10% faster at 9B, about 5% faster at 27B. - 4 concurrent requests at 27B: LM Studio 90.2 tok/s total, Ollama 33.1, because Ollama spilled part of the model to the CPU. - Disk: Ollama copies the GGUF; LM Studio can symlink it.
Not tested: Mac/MLX, ROCm, other quantizations.