Ask HN: Adaptive Spider Technologies?
Today a bot from 174.127.132.210 tried to copy information from my stealth-mode site (not linked to any domain name). This site was not supposed to be searchable and is known to a very limited number of people.
Any experience with such bots?
2 comments
[ 2.3 ms ] story [ 18.6 ms ] threadIt's generally not worth getting into an arms race against misbehaved bots, unless you're prepared to setup a honeypot or seriously analyze your access logs.