Scrapping HN / HNewsletter ?

1 points by lcrmorin ↗ HN
As a pet project, I would be interested in scrapping some queries on HN / HNewsletter. I am starting with Python (as per Automating the boring stuff with Python).

However, I encouter some practical problems.

For HN, I am able to search a term, but I am not sure how to navigate the different pages (or if it is needed at all).

How would one get all the titles (+points +number of comments) that match a specific query ? What would be the most practical way to also get the comments in an exploitable format ?

For hacker newsletter, I found that website : https://hackernewsletter.com/issues/ that seems to make the loop over issue relatively easy. But issue before 250 seems to be missing. Any idea on where I could find them ? Or an already compiled archive ?

2 comments

[ 2.7 ms ] story [ 11.7 ms ] thread
I believe a copy of HN data is on BigQuery, not sure how current