We ran the exact same Amazon shopping task with 9 leading AI models in the browser. Same site, same steps, same environment. Only the model changed. A few things stood out:
1. Fastest model: 70 seconds
2. Slowest model: 340 seconds
3. Cost range: $0.03 to $1.04
4. Only 2 of 9 models picked the right product!
4 comments
[ 0.26 ms ] story [ 29.7 ms ] thread1. Fastest model: 70 seconds 2. Slowest model: 340 seconds 3. Cost range: $0.03 to $1.04 4. Only 2 of 9 models picked the right product!
Arguably the ones that got the laptop, assumed you wanted a laptop, and went against your instructions.
I think that the result would be much better if you told them what exactly do you want in plain text.