[–] HackerDalius 7y ago ↗ Can the techniques be applied for other scrapers ? Because in the article they are discussed in the context of Web Robots scraping platform. [–] webrobots 7y ago ↗ This is relevant to frameworks that use headless or normal browser and Javascript. Plus of course jQuery to generate $.ajax requests.
[–] webrobots 7y ago ↗ This is relevant to frameworks that use headless or normal browser and Javascript. Plus of course jQuery to generate $.ajax requests.
[–] akriukas 7y ago ↗ I don't get what these "done()" mean. But otherwise, the logic looks good. [–] webrobots 7y ago ↗ done() is specific to WR system. Essentially we do not know when a step is finished because it may do waiting from DOM element or Ajax requests. So a developer must place done() where he/she is sure that a step has completed.
[–] webrobots 7y ago ↗ done() is specific to WR system. Essentially we do not know when a step is finished because it may do waiting from DOM element or Ajax requests. So a developer must place done() where he/she is sure that a step has completed.
4 comments
[ 4.7 ms ] story [ 27.1 ms ] thread