Why CommonCrawl is a Disruptive Force in Big Data (myeverwrite.com) 6 points by edmarferreira 14y ago ↗ HN
[–] asharp 14y ago ↗ Wasn't the major problem with commoncrawl that most of the index data was too old to be useful?Besides, search index crawling is comparitively cheap compared to the data processing required to make it useful for actual search.
1 comment
[ 3.2 ms ] story [ 10.5 ms ] threadBesides, search index crawling is comparitively cheap compared to the data processing required to make it useful for actual search.