HN Explorer

Ask HN: Is API scraping more common now than page scraping?

4 points by thinkloop ↗ HN
Given that many/most sites are SPA-style now that make structured http requests to retrieve their data (after initial page load), does modern scraping target underlying api's now, or is it still centered around parsing markup? If the latter, why?

2 comments

[ 3.6 ms ] story [ 9.0 ms ] thread
gcatalfamo
There isn’t a general rule. It depends on the website architecture. If there is an API you can use it is better for you and less tolling on the website (given you know what you are doing), otherwise scraping it is.
kakapumar
doesn't have any sense to make manual crawling and parsing, having tools like crawlera or proxycrawl makes it super easy to get data in API style way. So why complicate?