HN Explorer

Show HN: Turn any website into an API HackMIT 2013 (ec2-54-211-79-18.compute-1.amazonaws.com)

36 points by danysantiago ↗ HN
Select anything on a website. If it is part of a table or a list, you get a URL which you can GET for a JSON representation of that list or table. Basically we are making an API for every website whether they like it or not.

12 comments

[ 2.6 ms ] story [ 40.0 ms ] thread
tweeeyjg
What was it like? I saw on the HackMIT website that firearms and weapons weren't allowed.
sbruchmann
So, what’s the advantage over YQL? http://developer.yahoo.com/yql/
moneyrich4
it looks like yql is only for some sites and you need the sites api key, and this is any site "whether they like it or not" and it takes 2 clicks.

cool project thought about doing this myself, gl!.

sbruchmann
No, you can use YQL for any site that you want and even combine multiple queries into a single one. I don’t see any reasons why this project needs to exist…
drakaal
"whether they like it or not"

Basically they admit that this is a scraping tool for doing things you shouldn't.

Companies that want to sell you data do. Companies that provide data for free to use how you like do. This is basically automating stealing.

I have really low moral standards but I can see the wrongness of this.

icebraining
I haven't read the page, but from the title, I assume it's a scraper

Sure, it can be used for copying data without permission. But not all data is hard to read because they don't want you to copy it. For example, I've built a scraper for a state-run website of company registrations. The data is public, just not in a machine readable format.

diziet
Surely you are better off doing this locally rather than sending the data to your aws server?
hack37
Source code is available.

See chrome_extension/background.js.

Change post url accordingly.

caffeinetocode
Nice! Just got the back end up and running on my OSX machine.pointed the chrome plugin to my localhost. works like charm.

Though this looks similar to eBay's ql.io(open source) , I feel this is much simpler.