Both apps involved email for magic link features, but then both apps genuinely insist on harvesting your email for instantdb before you can view them?. Gotta love living on the post-shame internet.
This type of “first shot success” is an easy test to run but not really indicative of how useful the tools are in practice. It’s more important to test how they navigate a large codebase, make reliable changes in line with existing code, not introduce random regressions, be fast, etc.
I am just starting to feel that GPT-5 is more hype.
Just a day before GPT-5 launch, I made a video about making a tool with agents, Claude Sonnet 4 and GitHub Copilot.
There was so much hype on the launch day - on how good GPT-5 is, how it gets the code right the first time, and how little direction it needs.
So I was compelled to try it again with GPT-5 preview available in Copilot.
And for some reason, it struggled to align with directions.
a. It would make its own decisions, misaligned with what I mentioned. I had to explicitly say "DO NOT DO this....". ( explicit instructions were followed.)
b. It did not complete tasks and moved forward. This is the same style I used with both GPT-4o and Claude Sonnet 4.
Also, it might be good at coding, but it does feel geeky - need to try more.
4 comments
[ 2.9 ms ] story [ 18.5 ms ] threadJust a day before GPT-5 launch, I made a video about making a tool with agents, Claude Sonnet 4 and GitHub Copilot.
There was so much hype on the launch day - on how good GPT-5 is, how it gets the code right the first time, and how little direction it needs.
So I was compelled to try it again with GPT-5 preview available in Copilot.
And for some reason, it struggled to align with directions.
a. It would make its own decisions, misaligned with what I mentioned. I had to explicitly say "DO NOT DO this....". ( explicit instructions were followed.)
b. It did not complete tasks and moved forward. This is the same style I used with both GPT-4o and Claude Sonnet 4.
Also, it might be good at coding, but it does feel geeky - need to try more.