> Local models are never going to be as powerful. I think this point should be obvious: all of the current frontier models (closed and open-weights) are far too big to run on anything but a full GPU cluster in a datacenter
Diminishing returns with respect to scale (a 10X larger model is typically not 10X better at any given task) has so far meant that, even when datacenter compute grows faster than individual compute, the gap in quality between hosted and local models has generally shrunk. There are still plenty of tasks where that extra gain in quality is noticeable, but I feel there are also an increasing number of "saturated" tasks where it really doesn't matter.
Which puts more focus on other factors. Local models are private, low-latency, work offline, and can be tinkered with to your liking - like changing the system prompt to avoid refusals. I would not trust a hosted model to classify my documents, for example.
3 comments
[ 3.0 ms ] story [ 7.4 ms ] threadYes, and PCs will never be as powerful as mainframes.
And smart phones will never be as powerful as PCs.
But which ones are more popular and used more for computing?
In other words, "power" isn't the only factor that needs to be considered.
Diminishing returns with respect to scale (a 10X larger model is typically not 10X better at any given task) has so far meant that, even when datacenter compute grows faster than individual compute, the gap in quality between hosted and local models has generally shrunk. There are still plenty of tasks where that extra gain in quality is noticeable, but I feel there are also an increasing number of "saturated" tasks where it really doesn't matter.
Which puts more focus on other factors. Local models are private, low-latency, work offline, and can be tinkered with to your liking - like changing the system prompt to avoid refusals. I would not trust a hosted model to classify my documents, for example.
Proven wrong empirically by Nvidia back in November: https://arxiv.org/abs/2511.21689
"On HLE, Orchestrator achieves a score of 37.1%, outperforming GPT-5 (35.1%) while being 2.5x more efficient."
Cute blog though