I wanted to understand where local models stand today compared to frontier models. What kind of hardware is needed? What are inference options? And what does local vs open-weight hosted on data centers look like?
According to my own benchmarks running local models in my custom harness, Qwen3.5:122b was already better than Haiku and about equal with Sonnet 4.5 on coding tasks. Qwen3.8:27b-gsq-iq3s is better than both of them and also very fast on local hardware (~50tk/s on GB10 128Gb VRAM).
2 comments
[ 2.5 ms ] story [ 13.2 ms ] threadThe harness is just as important as the model.