Good news. O3-mini-high didn’t convince me at being better on complex coding questions than sonnet 3.5. Anthropic seems to be taking the lead now. Gemini Flash 2.0 is still only third.
This tells you a lot in terms of real-world usage. Claude is relatively expensive + openrouter users are generally price-conscious, yet it still crushes everything else. If investors are demanding new models that'd be absurdly short-sighted, on the level of if Apple investors would suddenly want bi-annual iPhone releases instead of annual ones.
Imo Anthropic's continued stability issues are also just due to extreme demand. Very telling too that everyone continues to build their products on top of them despite far more downtime than OpenAI, clearly it's worth it.
I love how we generally hear very little from Anthropic and don't have to deal with the constant hype. And they still have the preferred coding model despite its age. Looking forward to this release. Hope it will be a good one!
They have the preferred model for almost anything that reasoning isn't a perfect fit for, and even on those tasks only O1 wins consistently, for >10x pricing and latency.
Even for reasoning-beneficial tasks you can just CoT Sonnet and it works well, just yesterday I had a task's performance strongly improved by it.
8 comments
[ 3.3 ms ] story [ 23.4 ms ] threadThis tells you a lot in terms of real-world usage. Claude is relatively expensive + openrouter users are generally price-conscious, yet it still crushes everything else. If investors are demanding new models that'd be absurdly short-sighted, on the level of if Apple investors would suddenly want bi-annual iPhone releases instead of annual ones.
Imo Anthropic's continued stability issues are also just due to extreme demand. Very telling too that everyone continues to build their products on top of them despite far more downtime than OpenAI, clearly it's worth it.
Even for reasoning-beneficial tasks you can just CoT Sonnet and it works well, just yesterday I had a task's performance strongly improved by it.