1 comment

[ 3.4 ms ] story [ 13.9 ms ] thread
why not use proxycrawl to do the scraping? that way you would avoid the blocks, especially for the google queries you do. I mean for your example it's ok but as soon as you start to get serious there will be problems with proxies