How? It is pretty clear that this website is only interesting to "hackers", and there are already a lot of people who visit the site. I think what makes HN different than Reddit or any other similar website is the rules on posting and the subsequent consequences for not following those rules. That is what encourages interesting, useful comments and prevents stupid comments like "you suck, kill yourself". As far as I can tell, the Google ranking has little to do with it.
Thanks! I don't why they don't limit Google, Bing crawlers to the homepage via robots.txt. Then crawling should not overload an already busy website. Honestly, it seems like an overreaction to me.
For what it's worth, pg said that "crawlers are disproportionately bad for HN's performance because HN is optimized to serve recent stuff, which is usually in memory" (at https://news.ycombinator.com/item?id=3277661).
9 comments
[ 3.0 ms ] story [ 35.8 ms ] threadInterestingly the HN article it links to is the one the other day discussing how Google paid Adblock Plus to get its ads white listed.
Every now and again there are threads posted here discussing the apparent 'HN decline' as the userbase grows.