Http://www.google.com/bot.html = this page no longer available (google.com) 1 points by gstar 17y ago ↗ HN
[–] gstar 17y ago ↗ Crikey - I thought Google of all places wouldn't stuff up the user-agent bot webpage.Google's user-agent is: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.htmlI mean, everyone has heard of Google - but it's very bad mannered not to have a page up about your crawler.
2 comments
[ 2.7 ms ] story [ 18.9 ms ] threadGoogle's user-agent is: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html
I mean, everyone has heard of Google - but it's very bad mannered not to have a page up about your crawler.