GTP Blind Voting: GPT-5 vs. 4o (gptblindvoting.vercel.app)

I feel like the questions are way too simple. 3B models may perform similar with this sort of questions.

It is striking how similar these answers are to each other, hitting the same points beat for beat in a slightly different tone.

19/20 for gpt5. All the answers were very similar though I mostly just felt like the tone and delivery was better.

It's the same model

There should be a “both” option, I often found both answers acceptable but sometimes I strictly preferred one and those get sadly watered down.

This is a voting about the tone and note the quality of the answers.

My understanding was that with GPT-5 you don't actually get the high quality stuff unless the system decides that you need it. So, for simple questions you end up getting the subpar response. A bit like not getting hot water until you increase the flow enough to trigger the boiler to start the heating.

Lately I enjoy Grok the most for simple questions, even if it isn't necessarily about a recent event. Then I like OpenAI, Mistral, Deepseek equally and for some reason never felt good about Gemini. Tried switching to Gemini Pro the last two months but I found myslef going back to ChatGPT's free mode and Grok's free mode. Cancelled yesterday and now happily back to ChatGPT Plus.

I got %80 GPT-5 preference anyway.

I took the test with 10 questions, and carefully picked the answer with more specificity and unique propositional content, that felt like it was communicating more logic that was worth reading, and also the answers that were just obviously more logical or effective, or framed better. I chose GPT-5 8 out of 10 times.

17/3 - GPT-5/4o

12/8 gpt-5/gpt-4o

I took the "Rank Models" and got GPT5 and Sonnet 4 tied at 25% each, Gemini and Grok close by and 4o in the dust.

But ... the advice (answers) was quite uniform. In more than a few cases I would personally choose a different approach to all of them.

It'd be fun to have a few Chinese models in the mix and see if the cultural biases show up.

I don't like this test, because the very first question I was present with, had both answers looked equivalently good. Actually they were almost the same, just with different phrasing. So my choice would be absolute random. It means, that end score will be polluted by random. They should have added things like "both answers good" and "both answers bad".

I've got 7 GPT-5, and 3 4o.

This doesn’t really work as you can tell there’s an underlying prompt telling the model to reply in one or two sentences. That doesn’t seem like a good way to display the strengths and weaknesses of a model except in situations where you want a very short answer.

I gravitated choosing the longer answer so my result was a preference for GPT5 responses

[dead]

7/10 GPT-5.

Does anyone ever get answers this short? What's the system prompt here? That may bias things a little.

Also it's GPT not GTP

I did 20 questions.

In 75% of the answers I picked GPT-5, that's a pretty strong result, at least when it comes to subjective preferences!

19/20 GPT-5. I’m impressed.

I chose 4 more often! Was trying to be honest about what I preferred.

What on earth are these questions? They don’t resemble any real use of an llm for work.

Huh, I got 9/10 for GPT-5, and I was pretty convinced I was picking 4o in several questions based on the style. Interesting!

The questions were pretty much unlike anything I've ever asked an LLM though, is this how people use LLMs nowadays?

strange questions. I don't think self-help advice and advice for social relationships should be judged based on how popular it is. A lot of very similar and generic answers. Got an equal split when I took it. 10 each