HN Explorer

Ask HN: Web crawler/data product ideas?

9 points by MechanicalTwerk ↗ HN
Is there a data set you would love to have but you just don't have time to compile?

Is there a data set that you currently compile manually that you can't live without?

Would it make your life easier if you could avoid opening all those browser tabs every day/week/month and instead purchase a neat file (csv/google spreadsheet/something else?) with everything you need? Which browser tabs would this eliminate?

9 comments

[ 3.1 ms ] story [ 31.3 ms ] thread
[deleted]
(comment deleted)
JamieLewis
Probably not what you are looking for, but I would love to see more real-time data curation - I would certainly be willing to hand over cash for some decently priced real time data feeds - stock tickers, sports events, raw traffic data, headlines, currency, flights - many of these are available in pull formats, but it would be great to get a push feed.
dalek2point3
one big market for you is researchers. if only i had a penny for each time some economist came to me asking for a list of xxx in yyy, where x could be anything from flight prices, to businesses, to startups and yyy could be city / state/ coutnry /world. The internet is really good for current snapshots, but what reserachers most often need is repeated snapshots of the same thing. that helps us analyze how things change over itme. if you're a data company, please, please dont overwrite your tables to "update". save the old data. its really really useful. for research. and more importantly, for you.
MechanicalTwerk
Thanks. Will definitely look deeper into this. Know of any good sites where researchers make such requests?