One of the most interesting aspects of working on this project was the strategy we used for refining our primitive city detection. We gathered pairs of article titles & bodies (hoping we'd later accurately scrape them with the readability.js library...) and set them up as test fixtures. While TDD seems to go out the window during hackathons, it was crucial for us to quickly and efficiently iterate while making sure to catch as many edge cases as possible.
In typical hackathon fashion, the code got mangled when I had to switch the data store from leveldb (node) to flat json files (chrome extension) so the latest version in the repo isn't the most graceful. But here's a hacky node app for processing cities1000 data and then running mocha tests against a set of articles:
Extracting what location a news article is about is fascinating work. Unlike geocoding you don't get a fixed query.
The only commercial solution I'm aware of is Yahoo!s PlaceSpotter (middle of the page https://developer.yahoo.com/boss/geo/). Yahoo uses it to better target ads against content, e.g. news articles. I'm sure Google has something similar in-house.
8 comments
[ 2.1 ms ] story [ 28.5 ms ] threadIn typical hackathon fashion, the code got mangled when I had to switch the data store from leveldb (node) to flat json files (chrome extension) so the latest version in the repo isn't the most graceful. But here's a hacky node app for processing cities1000 data and then running mocha tests against a set of articles:
https://github.com/dzhang50/rlt/tree/master/node
Hope someone finds the technique and code useful, interesting, or at least amusing.
The only commercial solution I'm aware of is Yahoo!s PlaceSpotter (middle of the page https://developer.yahoo.com/boss/geo/). Yahoo uses it to better target ads against content, e.g. news articles. I'm sure Google has something similar in-house.