> Why your website is invisible to the agents visiting it.
I only _wish_ I could make a website invisible to agents. Instead now we're hostages to Cloudflare /shrug
Edit: so this isn't as "driveby-ish": the tldr; seems to be, if you want your site crawl-able, don't go overboard on the JS. This was good wisdom back then just as it is now in the current situation.
Fair point on llms.txt, but I think we're talking about two different layers here;
llms.txt helps agents find content -> API helps agents collaborate with a human
9 comments
[ 0.85 ms ] story [ 35.9 ms ] threadI only _wish_ I could make a website invisible to agents. Instead now we're hostages to Cloudflare /shrug
Edit: so this isn't as "driveby-ish": the tldr; seems to be, if you want your site crawl-able, don't go overboard on the JS. This was good wisdom back then just as it is now in the current situation.