> Multiple IPs makes some sense since requests are being made through random proxies, and I don't think on its own demonstrates intention of bad behavior. Agreed, just using multiple IPs isn't malicious on it's own. I…
Not only do they not respect robots.txt, but they publish an entire page[1] in their docs dedicated to circumventing scraping countermeasures. I pointed their scraper at a url on my server to test it's behavior. It made…
> Multiple IPs makes some sense since requests are being made through random proxies, and I don't think on its own demonstrates intention of bad behavior. Agreed, just using multiple IPs isn't malicious on it's own. I…
Not only do they not respect robots.txt, but they publish an entire page[1] in their docs dedicated to circumventing scraping countermeasures. I pointed their scraper at a url on my server to test it's behavior. It made…