[–] herpdyderp 1d ago ↗ I'm super bummed about this. FrontierCode 1.1 is my favorite benchmark and this has dramatically different results, I don't trust it at all! [–] Degorath 1d ago ↗ I sort of agree. Any benchmark that shows gpt-6 sol as better than gpt-5.6 sol is incredibly suspect.Or I'm wrong and working with a model has an aspect to it that is not captured by any benchmark.
[–] Degorath 1d ago ↗ I sort of agree. Any benchmark that shows gpt-6 sol as better than gpt-5.6 sol is incredibly suspect.Or I'm wrong and working with a model has an aspect to it that is not captured by any benchmark.
2 comments
[ 0.29 ms ] story [ 11.6 ms ] threadOr I'm wrong and working with a model has an aspect to it that is not captured by any benchmark.