I guess Netherlands.
Yep, that's why I've build my own, the existing ones don't give out a list of the links or are super slow. A co-worker made the first one in Python but it was so slow that it took hours (6+ sometimes) to finish a site…
I wrote a concurrent, super fast webcrawler for my job with Go (~300 LOC) to get data out of customer sites fast even when they have 1.5 million pages or more. You can basically filter everything to get a .csv file in…
I guess Netherlands.
Yep, that's why I've build my own, the existing ones don't give out a list of the links or are super slow. A co-worker made the first one in Python but it was so slow that it took hours (6+ sometimes) to finish a site…
I wrote a concurrent, super fast webcrawler for my job with Go (~300 LOC) to get data out of customer sites fast even when they have 1.5 million pages or more. You can basically filter everything to get a .csv file in…