I've done a lot of work in the scraping space for a side project I've worked on for years. At this point I've built my own tools, but over the years I've tested just about every paid scraping service that currently exists.
I think it would be really valuable to add some sort of tool that allows me to enter a URL and see a subset of the data returned, latency measurement, and flag if the Enhanced Bot Detection was required to get that data.
I can't count the number of times I've signed up for a service to see if they could get data from a URL that's giving me problems only to find that they couldn't scrape it either. It would be really helpful to know before I sign up what I need to pay for and that your tool can actually get the data I want. While it may result in fewer sign ups for your site, I think it will result in the customer who do sign up being higher value customers and potentially reduce support burden.
6 comments
[ 2.5 ms ] story [ 19.6 ms ] threadI think it would be really valuable to add some sort of tool that allows me to enter a URL and see a subset of the data returned, latency measurement, and flag if the Enhanced Bot Detection was required to get that data.
I can't count the number of times I've signed up for a service to see if they could get data from a URL that's giving me problems only to find that they couldn't scrape it either. It would be really helpful to know before I sign up what I need to pay for and that your tool can actually get the data I want. While it may result in fewer sign ups for your site, I think it will result in the customer who do sign up being higher value customers and potentially reduce support burden.
Bold claim.