2 comments

[ 3.2 ms ] story [ 14.7 ms ] thread
Crazy stuff. Why would anyone use headless browsers to scrape though, when you can just do requests directly to the endpoint.
Author here: most websites that have valuable data targeted by scrapers implement some sort of anti bots mechanisms. In particular they will force the user to execute different JS proof of works and run different browser fingerprinting tests to verify the nature of the browser.