Most browsers have safe browsing features that send unknown URLs to a cloud service for verification. I was shocked once to see an S3 presigned URL be accessed from Russia and India, when I had not shared it yet. It turned out to be Google safe browsing service (this case was several years ago and was verified with Google; I don’t know where URLs are investigated these days).
Another possible culprit is browser extensions. If you give an extension permission to a URL or page content, you have no idea if or where it’s sending it. Some extensions intentionally send URLs to cloud services for benign purposes (ex: Microsoft safe browsing extension for non-MS browsers).
I imagine this is Crawler Hints (https://blog.cloudflare.com/tag/crawler-hints/). If OP has Crawler Hints enabled on their domain then Cloudflare will let search engines know about pages.
> If you care about your unlisted URL privacy with Cloudflare, make sure you don't have the Crawler Hints feature enabled. - A good hunt, thanks and sorry!
How about not using CloudFlare at all?
Quoting from CF's own blog [0]:
> Going forward, because the benefits are so substantial for site owners, search operators, and the environment, we plan to start defaulting Crawler Hints on for all our customers.
Which by the way directly contradicts what jgrahamc stated in this thread.
12 comments
[ 39.4 ms ] story [ 47.2 ms ] threadDocs imply that this is not on by default [0]
[0] https://developers.cloudflare.com/cache/about/crawler-hints/
Another possible culprit is browser extensions. If you give an extension permission to a URL or page content, you have no idea if or where it’s sending it. Some extensions intentionally send URLs to cloud services for benign purposes (ex: Microsoft safe browsing extension for non-MS browsers).
If you cant find it this url may help (did not test): https://www.jezweb.com.au/incorrect-urls-in-bing-indexnow-fr...
[1]: https://developers.google.com/search/docs/crawling-indexing/...
> If you care about your unlisted URL privacy with Cloudflare, make sure you don't have the Crawler Hints feature enabled. - A good hunt, thanks and sorry!
How about not using CloudFlare at all?
Quoting from CF's own blog [0]:
> Going forward, because the benefits are so substantial for site owners, search operators, and the environment, we plan to start defaulting Crawler Hints on for all our customers.
Which by the way directly contradicts what jgrahamc stated in this thread.
[0] https://blog.cloudflare.com/crawler-hints-supports-microsoft...