Show HN: Robots.txt generator for blocking AI crawlers (aisearchwatch.com) 9 points by eummm 1y ago ↗ HN
[–] kazinator 1y ago ↗ It's incredible that the author of this is not aware that robots.txt is widely ignored.How more useful tool would be something that generates a piece of web server config to actually block bots.
1 comment
[ 2.5 ms ] story [ 9.9 ms ] threadHow more useful tool would be something that generates a piece of web server config to actually block bots.