Pretty wild! I wonder how much high school teachers and college professors are struggling with the inevitable usage though?
"Do deep internet research and thinking to present as much evidence in favor of the idea that JRR Tolkein's Lord of the Rings trilogy was inspired by Mervyn Peake's Gormenghast series."
Idea: workshops for teachers that teach them some kind of Socratic method that stimulates kids to support what they got from G with their own thinking, however basic and simple it may be.
Formulating the state of your current knowledge graph, that was just amplified by ChatGPT's research might be a way to offset the loss of XP ... XP that comes with grinding at whatever level kids currently find themselves ...
the thing about students who cheat is most of them are (at least in the context of schoolwork) very lazy and don't care if their work is high quality. i would guess waiting multiple minutes for Thinking mode to give thorough results is very unappealing. 4o or 4o-mini was already good enough for their purposes.
Slightly off topic but chatGPT’s refusal to visually identify people, including dead historical personalities, has been a big let down for me. I can paste in an image of JFK and it will refuse to tell me who it is.
Yeah, the % of the time I need to dip into deep research with GPT5 is much lower than GPT4 for sure. It even beats Gemini's web grounding which is impressive, I think most of the lift here is how smart/targeted its queries and follow-ups are.
These answers take a shockingly long time to resolve considering you can put the questions into Brave search and get basically the same answers in seconds.
The thing is, with Chat+Search you don't have to click various links, sift through content farms, or be subject to ads and/or accidental malware download.
I like Brave but have found their search to be awful. The AI stuff seems decent enough, but the results populated below are just never what I'm looking for.
> FWIW Deep Research doesn’t run on whatever you pick in the model selector. It’s a separate agent that uses dedicated o‑series research models: full mode runs on o3; after you hit the full‑mode cap it auto‑switches to a lightweight o4‑mini version. The picker governs normal chat (and the pre‑research clarifying Qs), not the research engine itself.
I do miss the earlier "heavy" models that had encyclopedic knowledge vs the new "lighter" models that rely on web search. Relying on web search surfaces a shallow layer of knowledge (thanks to SEO and all the other challenges of ranking web results) vs having ingested / memorized basically the entirety of human written knowledge beyond what's typically reachable within the first 10 results of a web search (eg: digitized offline libraries).
Yes, "GPT-5 with thinking" is great at search, but it's horrible that it shows "Network connection lost. Attempting to reconnect..." after you switch away from the app for even just a few seconds before coming back.
It's going to take a minute, so why do I need to keep looking at it and can't go read some more Wikipedia in the mean time?
This is insanely user hostile. Is it just me who encounters this? I'm on Plus plan on Android. Maybe you don't get this with Pro?
I've also found it to be good at digging deep on things I'm curious about, but don't care enough to spend a lot of time on. As an example, I wanted to know how much sugar by weight is in a coffee syrup so I could make my own dupe. My searches were drowned out by marketing material, but ChatGPT found a datasheet with the info I wanted. I would've eventually found it too, but that's too much effort for an unimportant task.
However, the non-thinking search is total garbage. It searches once, and then gives up or hallucinates if the results don't work out. I asked it the same question, and it says that the information isn't publicly available.
I find ChatGPT to be great at research too-but there are pathological failure modes where it is biased to shallow answers that are subtly wrong, even when definitive primary sources are readily available online:
I agree with this completely; ChatGPT search is perfect for most use cases. I find it to be better than OpenAI's deep research in my experience-- it often uses 2-3x the sources, and has a more comprehensive, well-thought-out report. I'm sure there are still cases where deep research is preferable, but I haven't come across those yet.
It really is great. When I was still on Reddit, I made regular use of the "Tip of My Tongue" sub to track down obscure stuff I half-remembered from years ago. It mostly worked, but there were a few stubborn cases that went unsolved, even after pouring every ounce of my Google Fu into the endeavor. I recently took the text of these unsolved posts and submitted them to Deep Research -- and within an hour, it had cracked four of them, and put me on track to find a fifth myself. Even if the reasoning part isn't entirely up to par, there's still something really powerful about being able to rapidly digest dozens of search results and pull out relevant information based on a loose description. And now I can have that kind of search power on demand in just a few minutes, without having to deal with Reddit's spambots and post filters and hordes of users who don't read the question or follow the sub's basic rules.
Yeah this is what people are doing with LLMs every day. I don't quite get what is supposed to be different in the blog post.
HN is a bit weird because it's got 99 articles about how evil LLMs are and one article that's like "oh hey I asked an LLM questions and got some answers" and people are like "wow amazing".
Not that I mind. I assume Simon just wanted to share some cool nerdy stuff and there's nothing wrong with the blog post. It's just surprising that it's posted not once but twice on HN and is on the front page when there's so much anti-AI sentiment otherwise.
I agree with Simon’s article but I usually think about “research” to mean comparing different kinds of evidence (not just the search part). Like evidence for the effectiveness of Obamacare. Or how some legal case may play out in the courts. Or how much The Critic influenced The Family Guy. Or even what the best way to use X feature of Y library.
I’ve found ChatGPT and other LLMS can struggle to evaluate evidence - to understand the biases behind sources - ie taking data from a sketchy think tank as gospel. I also have found in my work the more reasoning, the more hallucination. Especially when gathering many statistics.
That plus the usual sycophancy can cause the model to really want to find evidence to support your position. Even if you don’t think you’re asking a leading question, it can really want to answer your question in the affirmative.
I always ask ChatGPT do directly cite and evaluate sources. And try to get it in the mindset of comparing and contrasting arguments for and against. And I find I must argue against its points to see how it reacts.
50 comments
[ 3.0 ms ] story [ 73.5 ms ] thread"Do deep internet research and thinking to present as much evidence in favor of the idea that JRR Tolkein's Lord of the Rings trilogy was inspired by Mervyn Peake's Gormenghast series."
https://chatgpt.com/share/68bcd796-bf8c-800c-ad7a-51387b1e53...
Formulating the state of your current knowledge graph, that was just amplified by ChatGPT's research might be a way to offset the loss of XP ... XP that comes with grinding at whatever level kids currently find themselves ...
A while ago I bragged at a conference about how ChatGPT had "solved" something... Yeah, we know, it's from Wikipedia and it's wrong :)
This may nudge me to start using chatbots more for this type of queries. I usually use Perplexity or Kagi Assistant instead.
Simon, what's your opinion on doing the same with other frontier systems (like Claude?), or is there something specific to ChatGPT+GPT5?
I also like the name, nicely encodes some peculiarities of tech. Perhaps we should call AI agents "Goblins" instead.
[1] https://simonwillison.net/2025/Sep/6/research-goblin/
> FWIW Deep Research doesn’t run on whatever you pick in the model selector. It’s a separate agent that uses dedicated o‑series research models: full mode runs on o3; after you hit the full‑mode cap it auto‑switches to a lightweight o4‑mini version. The picker governs normal chat (and the pre‑research clarifying Qs), not the research engine itself.
Navigating their feature set is… fun.
It's going to take a minute, so why do I need to keep looking at it and can't go read some more Wikipedia in the mean time?
This is insanely user hostile. Is it just me who encounters this? I'm on Plus plan on Android. Maybe you don't get this with Pro?
Here's a screenshot of what I mean: https://imgur.com/a/9LZ1jTI
However, the non-thinking search is total garbage. It searches once, and then gives up or hallucinates if the results don't work out. I asked it the same question, and it says that the information isn't publicly available.
https://www.fortressofdoors.com/researchers-beware-of-chatgp...
Your Exeter cavern quandary was not exactly sorted. https://simonwillison.net/2025/Sep/6/research-goblin/#histor...
They are quite old and very well documented, so how on earth could a LLM fuck up unless, a LLM is some sort of next token guesser ...
The results look reasonable? It’s a good start, given how long it takes to hear back from our doctor on questions like this.
HN is a bit weird because it's got 99 articles about how evil LLMs are and one article that's like "oh hey I asked an LLM questions and got some answers" and people are like "wow amazing".
Not that I mind. I assume Simon just wanted to share some cool nerdy stuff and there's nothing wrong with the blog post. It's just surprising that it's posted not once but twice on HN and is on the front page when there's so much anti-AI sentiment otherwise.
I’ve found ChatGPT and other LLMS can struggle to evaluate evidence - to understand the biases behind sources - ie taking data from a sketchy think tank as gospel. I also have found in my work the more reasoning, the more hallucination. Especially when gathering many statistics.
That plus the usual sycophancy can cause the model to really want to find evidence to support your position. Even if you don’t think you’re asking a leading question, it can really want to answer your question in the affirmative.
I always ask ChatGPT do directly cite and evaluate sources. And try to get it in the mindset of comparing and contrasting arguments for and against. And I find I must argue against its points to see how it reacts.
More here https://softwaredoug.com/blog/2025/08/19/researching-with-ag...