8 comments

[ 0.29 ms ] story [ 20.6 ms ] thread
Not related to this release, but is anyone aware of what's happening with Deepseek? The usual cascade of synced releases has been lacking this frontier lab whale for a while now.
Quite a big improvement in coding benchmarks, doesn’t seem like progress is plateauing as some people predicted.
How should one compare benchmark results?

For example, SWE-bench Pro improved ~11% compared with Opus 4.6. Should one interpret it as 4.7 is able to solve more difficult problems? or 11% less hallucinations?