Hey all. Inspired by another Show HN a couple weeks ago, and also as a way to showcase what I can do while I looking for dev / data engineering / data science jobs, I built this app looks for new comments and threads on reddit that have links to amazon products and when I find one, display it on the site. Site itself is a Rails app, and the scraper is written in Python.
I don’t have too much data now, it’s only been running for something like 2 weeks, and I haven’t gone back in time yet either, but still fun to look and see what’s being talked about on random subreddits, and I really do see some bizarre product links. Next steps could be looking for other links, like youtube video mentions, or even more general NER trying to get products that aren’t linked, or scraping other sites for Amazon mentions, like here on HN. Let me know what you think.
2 comments
[ 4.1 ms ] story [ 17.2 ms ] threadRails app repo: https://github.com/jackschultz/productmentions Scraper repo: https://github.com/jackschultz/product-mentions-scraper
I don’t have too much data now, it’s only been running for something like 2 weeks, and I haven’t gone back in time yet either, but still fun to look and see what’s being talked about on random subreddits, and I really do see some bizarre product links. Next steps could be looking for other links, like youtube video mentions, or even more general NER trying to get products that aren’t linked, or scraping other sites for Amazon mentions, like here on HN. Let me know what you think.