Twitter bans unauthorized search engines from expanding shortened links (t.co)
Content of robots.txt is:
User-agent: * Disallow: /
No robots should try to expand twitter shortened links. So search engines have to pay Twitter (for firehose, for instance) to get better results.
0 comments
[ 2.6 ms ] story [ 11.1 ms ] threadNo comments yet.