[Error, corrected below] ~~The author is making incorrect assumptions about how ChatGPT works~~. ChatGPT simply does not go to specific websites suggested by users. It will perform Bing searches and "go" to those websites (or read from Bing cache?). I suspect navigating to specific websites suggested by end users could open up a variety of security concerns, AI safety concerns, and fraud concerns.
Yeah you're totally right, and tbh that's on openAI's side: they have to make it more clear that the system's agents aren't crawling a website in real-time. Unfortunately they don't make it clear and so people get confused like here
ChatGPT absolutely can visit websites provided by users. I proved that to myself just now by giving it a URL to a site I own, telling it to "Summarize URL", tailing the log file and confirming that the summary it returned included information that I had published on that page seconds before I prompted it: https://chat.openai.com/share/34f47777-59c8-47b0-9220-a1ce9b...
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot
Generally, if you see it say "Browsing..." it means it's either running searches on the Bing index or actually visiting pages. I don't think it's possible to tell which though, which is annoying.
Not sure if this is still possible, but before they introduced the GPT marketplace and were doing the plugins thing, it was possible to prompt inject ChatGPT by including a payload in the HTML of a site you control and asking it to summarise the page. I'm pretty amused at the GPs hope that something like this wouldn't be possible because it would create security concerns, because, yes, exactly! We're going to see a lot of SSRFs, file read vulns, etc. as LLM assistants are baked into more platforms. These things aren't magic and the same approach you would take to installing a WordPress plugin or some other third-party library must be used here. That said, given the attack surface plugins and libs continue to add to web apps, I'm not optimistic about this being solved before it becomes a problem.
"I'm sorry, but I can't browse external websites, including the one you provided. However, if you share some information or specific details from the page, I'd be happy to help you summarize or discuss them!"
It simply doesn't have the ability to visit a specific URL or answer questions about it. For that reason + a couple more niceties I use https://olly.bot
As always with GPT it's a hard balancing act. There is probably an instruction to ensure it does not visit urls un-necessarily, otherwise every request out there would trigger a search on bing if only to get more up to date information. Like "what is the capital of France" "Well let me ask Bing, you never know".
Whatever protection they have is clearly overcorrecting in this instance.
14 comments
[ 4.2 ms ] story [ 43.3 ms ] threadBut... it's also often VERY resistant to visiting pages. It makes judgement calls based on the URL you pass it - it refused to visit a PostBin URL I set up just now, at least at first: https://chat.openai.com/share/dcacf056-4509-40e7-bc54-6b9690...
Eventually it did visit it - if you hit https://www.toptal.com/developers/postbin/b/1709008827487-07... you may see the request from it (if it's not expired), which included the following user-agent:
Generally, if you see it say "Browsing..." it means it's either running searches on the Bing index or actually visiting pages. I don't think it's possible to tell which though, which is annoying.Before they introduced ChatGPT browsing mode it was unable to browse the web but frequently acted like it could, causing much confusion. I wrote about that here: https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces...
3.5. Refuses 100% of all requests.
Point stands, the presence of a URL really debased it!