There's been a bunch of discussion of gpt-4 "getting worse"; I tried gpt-4 vs gpt-4-0613 (the latest release with function calling), and noticed that while the latest release is much faster, its generalized instruction following capability indeed declined compared to the original... But you can get it to regain some lost reasoning capability by using the new function call API. Kinda weird!
I redid the test several times! GPT4 consistently got it right, and gpt-0613 got it wrong every time.
Anecdotally gpt-0613 has generally been "worse" for me on non-function-calls (but equally-ish good and much faster on function calls), but I wanted a consistent test for it. Feel free to try and duplicate on the Playground though!
We desperately need more advances on open models. Having this space controlled by corps that decide what we can or can not use of the SOTA is very frustrating
8 comments
[ 0.20 ms ] story [ 26.3 ms ] threada 'correct' answer one day and an 'incorrect' answer later demonstrates nothing.
Anecdotally gpt-0613 has generally been "worse" for me on non-function-calls (but equally-ish good and much faster on function calls), but I wanted a consistent test for it. Feel free to try and duplicate on the Playground though!