We were tired with dealing with the often steep pricing on geocoding when you reach your daily free limit (e.g. Google Maps starts at $10k/year). So I built this service so I can use it myself and hopefully it would be useful for others too.
Yep! Unfortunately we will manually have to add support for each country (including getting data, normalizing it, etc.) which is quite some work. We're planning to add support for additional countries if demand is high enough.
Separating a street name into "street" and "suffix" is a baffling decision which probably has a few issues even in the US, and definitely won't work elsewhere.
Agreed, this is unusable for us until there's international support. We have a large percentage of international users in 130 different countries. Doesn't make sense to use this over, say, MapQuest.
The biggest problem we've had is changing non well-formed addresses / ambiguous addresses into canonical addresses with lat/lng. Google Maps wins on that front.
We obviously can't beat Google in that case :) That's also why it's priced to be way more affordable. It does however happen that Geocodio is more accurate than Google Maps - try for example "8895 Highway 29 South, 30646" (Address of a CVS store) on Google Maps and Geocodio.
I'm using mapQuest geocoding API[1] which basically does what you do for free, without the rate limits.
setting it up was quite a pain because they don't use semantic http codes, and I had to play with it a lot to handle their undocumented error codes (they store it inside body.info.statuscode). Good to read that you return semantic http codes.
If you want to differentiate from the competition, I would suggest that you improve the address parsing and support more patterns. Think of us having to geocode user-typed location fields from twitter. Enjoy it :)
Cherry picking one example does not make you more accurate than Google Maps. TIGER has some giant holes in it, and is based on block faces not building footprints like Google Maps. In most cases Google Maps will be much more accurate and comprehensive.
Thanks! Yes, I haven't really seen a lot of other services that provides batch geocoding as an API endpoint.
We have mostly been running tests against the Google Maps API, and from a totally random sample of 100 address, 90 of them were within a mile from the Google Maps API returned location (Most of them were actually within 0.01 mile).
I'm not sure how we would compare to OpenStreetMaps and Data Science Toolkit since our data source is different (US Census Bureau). - But the obvious reason why we provide this as a SaaS, is that you don't have to host anything yourself, or juggle around with gigabytes of boundary data. We handle all the mess.
Our infrastructure is pretty efficient, making us able to keep our operating costs low. We wanted to have a pricing point that was below any other similar services we could find.
I work at SmartyStreets, where we've learned that geocoding is very, very difficult, so I definitely feel your pain! We started with basic Census Bureau stuff and it's definitely complicated, and accuracy can be spotty. (We've since worked with other data vendors to improve the accuracy.) It's too bad we don't all have little cars to roam the country with and manually collect rooftop-level data like Google does.
+1 on the versioned API endpoint... when we released ours nearly 8 years ago, versioning APIs wasn't really a thing yet. We're paying that technical debt off now as we vigorously rewrite and improve our service.
Quick feedback: Links on the FAQ page are hard to distinguish from regular text.
Thanks! Yes, it is definitely not easy, a lot of edge cases to take care of too. Luckily we are not trying to directly compete with any of the big guys out there, which makes us able to keep the price low and the output high.
Good point, we might want add that as an optional parameter. Also note that our address address parsing API endpoint is free and doesn't count towards the usage statistics and billing :)
If there's enough interest, we'll definitely be working on this next. It would just require a slight restructuring of our data to make the lookups as efficient as possible.
+1 to reverse geocoding support. Our app runs around 25k/day reverse geocode calls to OSM and Mapquest's Nominatim. We are projecting up to 4x growth within the year so an accurate, bulk and cheap service will help ease our pain. And oh, we're based and operating in the Philippines (which hopefully you can add soon as well).
Where the pricing says $.001/ea for 2501+ geocodes, are the first 2500/day prior to that still free? Or am I paying $2.50 for the day as soon as I make that 1 extra request above the free limit?
UI suggestion: 'street addresses' currently has a box around it, so I thought it was an <input type="text"> field, thought "how cute", tried to click on it to enter an address to geocode, and was disappointed to find out it was just some bolded text. Might be a fun little feature to have that actually be an entry point into trying out a demo of the API (I thought I was supposed to enter an address to have geocoded).
I oversaw a project like this elsewhere (where we had reams and reams of geo coordinates, but we needed text searchable tags (like "Canada", "Toronto", etc).
We had millions of them though, so maybe an API isn't really the way to go.
An admirably quick response, both to my question here, and to the email I sent.
People, this is a lesson. If you post a "Show HN" then be ready to respond to people's questions and comments. Posting and then going silent for hours is not a good message to send
to people who you want using your service. It says you haven't thought enough about your level of service.
Gah! This is awesome. Where were you when I was trying to get an idea launched and the cost of geocoding was the wall I kept hitting??? Seriously this makes my week, maybe it's time to dust off some old projects...
See my previous answer [1], obviously it's impossible to compete directly with Google and especially not at this price point. Our goal is to return a geo coordinate that is at least on the correct block and as close to the street number as possible.
For those looking to roll your own, the Ruby implementation of a TIGER geocoder released by GeoIQ a while back is a pretty solid starting point: https://github.com/geocommons/geocoder/
We ended up using that as a base and then making some customizations for our US-based geocoding solution. As these guys are figuring out, there's no great int'l option. Google is bad from a licensing perspective (but their tech is fantastic). MapQuest is great but can get really expensive. We've had decent luck with TomTom I think, but if I remember correctly there are a lot of caveats.
That's a pretty nice starting point, but unfortunately the code base hasn't been updated for years and the data import process is extremely time consuming [1]. That said, rolling your own geocoding solution is the most restriction free service you can get. Just be prepared for the maintenance and the time consuming set up.
It's a very smart geocoder that one, have contributed to it - It is a bit old now and not that easy to get started with.
The state of the art of open source geocoder would be TwoFishes: https://github.com/foursquare/twofishes written in Scala and developed and used by FourSquare
I had the need to geocode 10s to 100s of thousands of US addresses weekly, with the ability to accept slightly-reduced accuracy vs. the parcel-level accuracy of Google Maps.
I rewrote the geocommons geocoder in Java to speed up the loading and geocoding process, and wrapped a REST api around it. I used a minimal perfect hash function to map zips/streets (metaphone3'd and ngramfingerprint'd) to data stored in a key-value structure. The key-value structure is small enough to fit in memory of a decent sized EC2 instance, but I haven't tested the throughput except from a slow disk--which got me about 100-150 results/sec.
The results include parsed address, lat/lng in WGS84 datum, and associated US census region info (state, county, block group, block, msa, cbsa/csa, school district, legislative district, etc.).
I'd considered open sourcing it, and I was trying to architect it such that one could plug in various data sources beyond TIGER when higher-accuracy info is available (e.g., from SF's address parcels, Massachusetts has lots of E911 parcel data available, etc).
I wonder how the "choose your own api key" policy is going to work in practice... given that people don't usually make very secure passwords and that the example is "Real estate website" you're going to get some pretty easy to guess api keys.
That's actually just a name to identify the API key, the actual API key is a 40-character automatically generated string. The idea is that you will be able to create an API key for each of your projects and revoke them individually as necessary.
103 comments
[ 3.4 ms ] story [ 170 ms ] threadThe biggest problem we've had is changing non well-formed addresses / ambiguous addresses into canonical addresses with lat/lng. Google Maps wins on that front.
setting it up was quite a pain because they don't use semantic http codes, and I had to play with it a lot to handle their undocumented error codes (they store it inside body.info.statuscode). Good to read that you return semantic http codes.
If you want to differentiate from the competition, I would suggest that you improve the address parsing and support more patterns. Think of us having to geocode user-typed location fields from twitter. Enjoy it :)
[1]: http://open.mapquestapi.com/geocoding/
I also wrote a primer explaining the basic geocoding ideas:
http://cjauvin.blogspot.ca/2012/04/lean-geocoding.html
How does the accuracy (as well as address parsing capabilities) compare to the completely free solutions such as Nominatim[1] or DSTK[2]?
Both services provide capabilities for local installations, obviously with no query limits and minimal latency.
[1] http://wiki.openstreetmap.org/wiki/Nominatim
[2] http://www.datasciencetoolkit.org/
We have mostly been running tests against the Google Maps API, and from a totally random sample of 100 address, 90 of them were within a mile from the Google Maps API returned location (Most of them were actually within 0.01 mile).
I'm not sure how we would compare to OpenStreetMaps and Data Science Toolkit since our data source is different (US Census Bureau). - But the obvious reason why we provide this as a SaaS, is that you don't have to host anything yourself, or juggle around with gigabytes of boundary data. We handle all the mess.
+1 on the versioned API endpoint... when we released ours nearly 8 years ago, versioning APIs wasn't really a thing yet. We're paying that technical debt off now as we vigorously rewrite and improve our service.
Quick feedback: Links on the FAQ page are hard to distinguish from regular text.
Good luck with the project!
We'll update the FAQ links, thanks!
Most of the times users will only care about the results, so you'll be sending useless data
Would actually be neat though for that to be a quick demo of your software.
We had millions of them though, so maybe an API isn't really the way to go.
People, this is a lesson. If you post a "Show HN" then be ready to respond to people's questions and comments. Posting and then going silent for hours is not a good message to send to people who you want using your service. It says you haven't thought enough about your level of service.
Kudos to GeoCod.io.
https://news.ycombinator.com/item?id=7095467
We ended up using that as a base and then making some customizations for our US-based geocoding solution. As these guys are figuring out, there's no great int'l option. Google is bad from a licensing perspective (but their tech is fantastic). MapQuest is great but can get really expensive. We've had decent luck with TomTom I think, but if I remember correctly there are a lot of caveats.
[1] https://github.com/geocommons/geocoder/wiki/Installation-Ins...
The state of the art of open source geocoder would be TwoFishes: https://github.com/foursquare/twofishes written in Scala and developed and used by FourSquare
I rewrote the geocommons geocoder in Java to speed up the loading and geocoding process, and wrapped a REST api around it. I used a minimal perfect hash function to map zips/streets (metaphone3'd and ngramfingerprint'd) to data stored in a key-value structure. The key-value structure is small enough to fit in memory of a decent sized EC2 instance, but I haven't tested the throughput except from a slow disk--which got me about 100-150 results/sec.
The results include parsed address, lat/lng in WGS84 datum, and associated US census region info (state, county, block group, block, msa, cbsa/csa, school district, legislative district, etc.).
I'd considered open sourcing it, and I was trying to architect it such that one could plug in various data sources beyond TIGER when higher-accuracy info is available (e.g., from SF's address parcels, Massachusetts has lots of E911 parcel data available, etc).