HN Explorer

Maps.earth – free and open-source web maps (about.maps.earth)

696 points by ellenhp ↗ HN
Hey everyone, a while back I posted about an open-source maps project I was working on called Headway. I just wanted to share an exciting update. I made a few architectural improvements with the help of some really awesome contributors and Headway can now run full-planet instances. I'm really excited with what we've managed to come up with.

Right now the full-planet demo server is in Germany and lives at [1]. A brief explanation is at [2]. I'm hopeful that the project will continue to improve with time and eventually end up being a real contender in the web maps space. I'm talking with folks about the potential for getting some more robust hosting options too which is very exciting.

For the moment though, Headway has some usability issues. It's missing ETAs, has no steps list for directions, or proper geocoding search results page, etc. I'm going to get to these issues with time, but please drop me a line if you're interested in helping out. Especially interested in help with web frontend, design, or internationalization. Development takes place on GitHub [3] and discussions tend to happen in the matrix room [4].

[1] https://maps.earth/

[2] https://about.maps.earth/

[3] https://github.com/headwaymaps/headway

[4] https://matrix.to/#/#headway:matrix.org

116 comments

[ 3.5 ms ] story [ 185 ms ] thread
[deleted]
(comment deleted)
abetusk
I'm a little confused as to what this actually is.

Is this a route planning built on top of OSM? Does this bundle OSM data "all in one" so that you can more easily host an OSM server?

ellenhp
OSM doesn't have a server per se (other than the OSM API server). It's a dataset for the most part. Headway lets you bring up a server that will render maps tiles, allows you to search for places and get routes between places. It's trying to be a user-friendly UI for various pieces of the OSM ecosystem. There are a lot of amazing pieces of software out there that work with OSM data, but there isn't an easy way to bring them up, and there's no unified frontend for all of them other than Headway (that I know of).

Edit: To the comments telling me I'm wrong about this, remember that Headway has a geocoder and routing system, along with a transit trip planner (currently disabled on maps.earth though). These are not things you could get with off-the-shelf solutions that include a GUI until now. I also want to self-host my mapping software for privacy reasons, and Headway makes that easy too. I probably need to be better about explaining what all Headway does that's different than just bringing up an OSM tileserver though. Thank you all for the feedback :)

chess_buster
Thanks.

It would be great if this could be stated on the page.

I kept hackernews open in order that someone would explain to me what this actually does. Your comment thankfully fullfilled this.

ellenhp
Thank you for the feedback :)
soheil
There no frontend to bring up an OSM? Literally you need a single line to load one up:

  (new map('map_div')).addLayer(new TileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png'))
simon04
Using tile.openstreetmap.org is subject to the OSM tile usage policy: https://wiki.openstreetmap.org/wiki/Tile_usage_policy
ellenhp
To add onto this, those are raster tiles and the industry is moving away from raster tiles for a multitude of reasons. Headway doesn't bulk-download tiles from OSM, by default it takes an OSM extract from BBBike (with explicit permission) and brings up a server based on that. It generates vector tiles with planetiler and serves them with tileserver-gl-light. I've put a lot of thought into the way it all works in an effort to minimize load on the OSM ecosystem's infrastructure, provide attribution where required, etc.

I can promise you that there's no one-line replacement for the work I've done here unless you want to go out and pay for a closed-source product.

abetusk
There are services hosting OSM data, often times requiring paid API access for anything other than personal use.

OSM is amazing, for sure, but standing up even a small OSM service that only services a map in your local town/city/community is too daunting for most people (myself included).

Considering there are many other comments in this thread that mirror my main question, I would try and figure out messaging. Is it a particular tile rendering on top of OSM data? Is it trip heading information? Is it helping people stand up self hosted OSM servers? Is it helping with just a few components of that process? If so, which components? etc.

It's hard to know what the "value add" is here and a lot of this is my own ignorance on how the OSM ecosystem works but that's kind of what I would recommend trying to convey to people.

I wouldn't assume deep familiarity with OSM. I would like to hear more, especially concerning tools to help stand up a self hosted OSM server more easily, as I would suspect many other people would, but it would be nice to have some basics spelled out instead of assuming I have deep familiarity with the sprawling and intricate OSM ecosystem.

Aeolun
> tools to help stand up a self hosted OSM server more easily

I think that’s already the wrong way to think about it. For some reason that I do not comprehend, OSM comprises at least 3 different file formats, and 5 or 6 different ways to store that data.

When you ‘set up an OSM server’, what you really do is pick one of the storage methods, and then import the data with one of a variety of tools that were built over the years (and are probably compatible with only one or two of the storage methods each).

Seriously, the OSM ecosystem is a clusterfuck. Anyone that aims to standardize it into something easy to deploy has my full support.

stevage
I built something along the same lines many years ago, with routing but not geocoding.

https://github.com/stevage/saltymill

Yeah, it's a pain in the arse getting all the bits together. And I wasn't attempting full planet scale.

abetusk
OK, the GitHub page spells it out better [0].

> Self-hostable maps stack, powered by OpenStreetMap.

In addition to listing out some of the required specs (for a localized area) [1]:

> ... generation ... needs ... 8GB of memory ... running ... around 4GB ...

> ... recommend ... 50GB ... disk space

[0] https://github.com/headwaymaps/headway

[1] https://github.com/headwaymaps/headway#system-requirements

2Gkashmiri
could we.... uh, have our server for a specific city or town for example if the use case is around that area only? that would save a lot of resources
ellenhp
Yup! That's how most people run Headway right now. You can probably get by with around 2GB of RAM for a medium-sized metro, possibly even less.

Also for full-planet requirements, the OVH box running maps.earth has 64GB of RAM and 6 physical cores and is really struggling right now. Load average is about 40, most its CPU time is spent on the Valhalla service. Really enjoying watching the metrics right now :)

2Gkashmiri
could you put up munim or other similar metrics site for public? it would be nice to see such a high load
zylepe
You may have chosen Valhalla for other reasons, but FWIW I’ve found Graphhopper can handle 1-2 orders of magnitude more qps than Valhalla on the same hardware.
ellenhp
I wasn't happy with cycling directions in GraphHopper and I was having trouble getting route shapes from its routing endpoint which made it really difficult to plot route polylines on the map. I've also been looking forward to exposing some customization of routing parameters since valhalla allows you to do this fairly trivially. That is good to know, though. Valhalla really is getting hammered by all this traffic. :/
karussell
What problems did you face with cycling directions?

Regarding the route shapes I can only guess: we use an encoded polyline as default for a very compact response. You can either disable this or use our JavaScript library, or there is a new UI https://github.com/graphhopper/graphhopper-maps

Regarding routing parameters: our custom_model approach is more powerful than simple parameters and still not complicated: https://www.graphhopper.com/blog/2020/05/31/examples-for-cus... See e.g. the several bike examples and see how to get an world wide instance up and running: https://www.graphhopper.com/blog/2022/06/27/host-your-own-wo...

In recent master you can additionally control how elevation influences your route (e.g. prefer or avoid them entirely or steep sections or similar)

DonnyV
Didn't have my address in NJ.
tuukkah
You should be able to add it at https://openstreetmap.org :-)
jmkb
But don't expect it to show up on maps.earth right away... it looks like it's at least a month behind OSM at this point.
ellenhp
That's mostly because I decided to use maps from the daylight maps distribution. The current tiles were actually built last night. I think it probably makes sense to track planet.osm.pbf directly though because I want to create a virtuous cycle where users feel that contributing to OSM has a real tangible payoff.

The main issue with updating regularly is going to be that long-term I can't really afford a beefy enough machine that has a fast uplink to Cloudflare where my artifacts are hosted. Getting all of the planet-sized artifacts uploaded even just this one time was a huge pain. I ended up having to generate them on my desktop, ferry the data to a friend's house on a laptop via motorcycle and upload them from there. This is one of the things that I want to help remedy if I get enough support via Liberapay or GH Sponsors.

Aeolun
It doesn’t pull the OSM data (from their servers)? Or you did that once and pre-processed everything now?

I presume you can do something similar for the daily and minutely updates, but I imagine merging that with the existing stuff is more work.

tuukkah
Have you explored whether it would be possible to upload only the changed parts?
paxys
I tried the hosted app at https://maps.earth but it isn't really usable at present. How do you actually search for and navigate to places?
ellenhp
The search box in the top left should bring up a bunch of autocomplete suggestions when you type stuff into it. Clicking on one brings up a place card view [1]. If my demo server falls over that will fail to populate. Once you have a place card you can hit "bike there" or something and then populate the "From" field that appears again in the top left to get a directions view [2]. Not ideal, the frontend was built in a few weeks by someone with very little frontend experience (me) so I did my best.

[1] https://maps.earth/place/openstreetmap%3Avenue%3Away%2F12903...

[2] https://maps.earth/directions/bicycle/openstreetmap%3Avenue%...

paxys
I guess it may be failing due to the request spike. Even otherwise, searching for common places like "San Francisco" or "New York" shows a dozen results all over Europe and South America, not the one place most people would be looking for. Street address search doesn't seem to work either.
owenpalmer
Not working for me either. No results show up, even for common places like LA.
ellenhp
Yeah, I've noticed that at least the way Pelias is configured in Headway, searches for administrative areas don't seem to work, like at all. I'm not sure if that's a config issue or an issue with Pelias though. Address search should definitely work as long as there's an appropriately tagged OSM node for your address, but the server's load average is 40 on a 6 core OVH box in Germany so you might have to be (very) patient to get anything back until the load drops to a level it can deal with.
juliansimioni
Hi! Maintainer of Pelias here and co-founder of Geocode Earth where we continue to fund Pelias development through our SaaS and consulting.

You need to bring Who's on First data in to support administrative areas, it won't use admin areas from OSM. This is a bit of a technical choice (WOF has some nice features like stable IDs and a rich hierarchy, and a data format that's much easier to work with), but also the result of the Who's on First project and Pelias being developed together while we were at Mapzen.

Don't hesitate to reach out to us if you need help getting things set up.

ellenhp
Is there a contact point you'd prefer me to use for reaching out? I'm pretty sure I'm using WOF for bringing up Pelias but it's entirely possible I'm configuring things wrong. I'm also having a lot of trouble getting good performance out of the viewport biasing on the autocomplete endpoint (`focus.point`). Photon has `location_bias_scale` which allowed me to do very strong viewport biasing for zoomed-in queries and eliminate it almost entirely when zoomed out.

Is this something that could be added? I don't mind doing a bit of legwork if need be. This is important enough that I'm willing to throw a few weeks at implementing it.

The pelias.json for this instance is at: https://publicdata.ellenhp.workers.dev/planet-v1.16.pelias.j...

I'll go double check that the placeholder container is operating correctly, actually. :)

ElCheapo
I just asked for directions from my home to the railway station and the navigator told me to enter a roundabout in the opposite direction :/
uneekname
The direction of that roundabout might be mis-tagged on OpenStreetMap. Would you be comfortable sending me the location, so I can see about fixing it?
[deleted]
(comment deleted)
ElCheapo
Forget that, I erroneously asked for walking directions my bad. Shame I can't delete the comment lol
soheil
Can you tell us what this even is before telling us how sorry you are that it’s not perfect?
ellenhp
It's a self-hostable web maps stack trying to be an open-source alternative to Google Maps.
[deleted]
(comment deleted)
playingalong
I like the fact it uses Earthly for building. Nice pun.
joekrill
Wow this is pretty amazing. For those that haven't tried, it's really difficult to piece this sort of thing together. So the fact that this is all provided as a Docker Compose configuration is really awesome.
shafyy
Hey, this looks great, thanks for working on it! I'm the creator of Mapzy[0], an open-source and self-hostable store finder. We're trying to focus on privacy as much as possible, but currently use Mapbox since it's better than Google Maps and we didn't find any other alternatives.

Do you think Headway could be a replacement for us? We currently use the Mapbox JS library to create the pins etc. - could we use Headway with Leaflet JS for example? We also use the Mapbox geocoder, mainly for reverse geocoding of addresses. How does Headway's geocoder compare?

0: https://github.com/mapzy/mapzy/

jutaz
I’d recommend taking a look at Maplibre GL JS[0], which is a fully open source map library. It was forked from Mapbox GL JS just before licence change, therefore it’s also fully compatible with the style spec etc.

It’s also actively maintained and has a strong community behind it.

0: https://github.com/maplibre/maplibre-gl-js

twelvechairs
If you are in this space deck.gl [0] is well worth cehcking out. It does scale at speed, 3d (more kinds), motion and effects extremely well.

[0] https://deck.gl/

shafyy
Thanks, I'll check it out!
bearjaws
We're in the same boat using Mapbox, we've been debating leaving for the OpenStreetMaps container since we only use them for static images and map display. https://github.com/Overv/openstreetmap-tile-server

Our bill last year was $80k and honestly its right on the cusp of being worth the effort to switch.

gardnr
> Our bill last year was $80k

You might be able to pay someone from the project to help you switch. A nice way to give back and it will probably be a smoother transition.

Bedon292
Have you looked into Nominatim [1] for reverse geocoding?

[1] https://wiki.openstreetmap.org/wiki/Nominatim

ellenhp
Nominatim requires a lot of resources for full planet installs. I'd recommend looking into Pelias or Photon instead. They're both amazing. Headway has used both of them at various points. Photon is definitely easier to set up for the planet, though, you can just download a pre-computed search index from GraphHopper and you're good to go.
shafyy
I haven't. Thanks for the recommendation!
ellenhp
Headway was designed more for consumer use-cases than anything else. There's been some talk of adding the ability to customize it in the future but I don't have time for that because I need to focus on the basics first. You could fork it though, or just use some of the data automation (respecting the Apache 2.0 license). I replied to a child comment here but my recommendations for geocoding would be Pelias or Photon, and I like maplibre-gl-js. You can (should!) use planetiler for generating your mbtiles files and I've had generally good experiences with tileserver-gl-light, but many people recommend using PMTiles instead. It kind of just depends what your requirements are. For privacy-first stuff like Headway I can't require that people make requests out to S3 or Cloudflare, it has to be self-hostable which means a traditional tileserver is the only way to go.
shafyy
Thanks for the info! I'll look into your recommendations.
petre
Try MapTiler. It works fine with what you're using. For geocoding, I dunno, we run our own show.
shafyy
Thanks! MapTiler looks great, but a bit on the pricey side for our use case.
petre
We picked it specifically because Mapbox was too expensive. We pay like between 20 and 30 euros each month for tens of thousands of transactions. You can also buy tile sets and serve your own vector tiles, but it isn't worth the hassle for us.
shafyy
Have you ever compared it to https://github.com/Overv/openstreetmap-tile-server ?
petre
No. We no longer want to bother with monthly updates and the overhead of maintaining our own tile server over <30€/month. We tried to make our own raster tile server several years before vector tiles were a thing, using tilecache, then tilestache and just gave up because of the storage and hardware requirements. Plus there's a lot of work involved in customizing your maps - we didn't want to show the standard OSM map built for mappers in mind but a transport oriented map for fleet tracking. MapTiler has almost everything we want except good Africa coverage.
shafyy
Makes sense, thanks for your explanation.
NanoWar
You can spin up an own instance in a docker container easily (if you self host everything anyways).
shafyy
Yeah just saw that after making the comment, thanks!
alexalx666
Is it possible to access it via less js heavy frontend? Owner of Nokia N900 here :D Would great to have Ghost in the Shell style super lightweight maps
ellenhp
Unfortunately no. maplibre-gl-js requires webgl for vector tile rendering. That's how you get snappy zoom performance, reduce the load on the end-user's cellular connection, etc.

Sick phone though I'm a little bit jealous :)

alexalx666
I can also imagine requesting route via terminal and getting just list of directions which is many times enough
tuukkah
I have seen (but cannot find now) a command line tool that you give two addresses, it queries OpenTripPlanner for itineraries and displays the result using walking, train etc. Unicode emojis.
alexalx666
thanks! it works via REST/GQL apis so should be easy to write a cli
pessimizer
You wouldn't be as jealous if you knew how hard it is to find batteries these days.
alexalx666
I found a 1600mAh one from AkkuKing on German Amazon, I thought it will be harder,
codethief
Very impressive!

Is it just me (Firefox on Android) or does the UI really not allow tapping landmarks (shops, restaurants, subway stations)?

Also, directions don't seem to work right now (infinite loading) but I suppose this is due to the server receiving the HN hug of death right now.

By the way, for anyone looking for the mobile (non-web) version of this: Organic Maps is a fantastic frontend for OpenStreetMaps (and of course there's the good old OsmAnd, though I like Organic Maps better these days).

jonas-w
Thanks for Organic Maps!

The website doesn't seem to work for me either on mobile.

sorenjan
OsmAnd doesn't seem to get many updates nowadays. I get that it's open source and we can't demand anything for free and all that, but they have some monetization options that should bring in some money, but aren't really worth the price IMO. The issue tracker is full of bugs and feature requests marked as nice to have, but almost nothing happens.
oynqr
Recently switched from Osmand to Organic and am quite impressed. Rendering and routing are much faster and there are no crashes even with very long routes.
ssl232
Agreed, it's so much more pretty and the UI is slick. With Osmand I always seem to end up with stuff on the map from a previous search/navigation that I don't want any more, but can't figure out how to get rid of.
vanous
This is amazing, thank you very much! It is really tempting to spin up a small instance for hyper local maps.
ellenhp
If you do spin one up, feel free to join the matrix and share the experience or just email me with feedback. I'm trying to get this as frictionless as possible to set up because I want people to own their own location data.

Headway is much much more responsive on a local scale than it is in https://maps.earth/ and the search results are more relevant too. My own personal instance for Seattle has largely replaced Google Maps for me. :)

c0nsumer
This seems really neat. I have a question that I hope isn't something I'm missing, but is there an easy way to change the rendering ruleset? I ask because I've been wanting to build some custom maps for mountain bike trails in the state where I live and have them served up similarly.

Whenever I've looked into other projects for this it's either focused on rendering tiles from Mapbox data (where not all the OSM tags -- especially not MTBing ones -- are available) or really old info or presumes that someone else will be generating tiles.

ellenhp
This is tracked here: https://github.com/headwaymaps/headway/issues/85

It's going to be somewhat tricky because of a few mediocre decisions I made early on in the development process, but if you were up for helping out I'd love to get a bring-your-own-style experience working in Headway. Long-term, I want to add the ability to easily brand Headway and customize it for your own needs without forking it.

c0nsumer
Thanks for pointing me there. I'll give it a read and see what I might be able to do.
sorenjan
How do you handle map data updates? Do you regenerate everything from scratch from the full data set, or can you update it incrementally with patches? Is it possible to put the tiles on a CDN like cloudflare?
KoftaBob
Awesome work!

What are your thoughts on the feasibility/benefits of using a p2p system like IPFS to share OSM map/tile data between users, rather than serving from one centralized server?

ellenhp
It's... complicated. The discussion for it is here, I'd be curious if you had any thoughts in particular about this. Feel free to comment or join the headway Matrix room if you want to chat more in-depth. In principle I'm open to this but the privacy implications are a little weird. https://github.com/headwaymaps/headway/issues/50
[deleted]
(comment deleted)
ajsnigrutin
How much resources does something like this use? (full planet, few (<10) users)? Disk, RAM, cpu cores?
ellenhp
I cover that in the readme, but the short answer is that the server running maps.earth has 64GB of RAM and 6 physical cores. It's using about 600GB of disk space, but the disk usage might change because I suspect my Pelias index is missing a lot of information.

https://github.com/headwaymaps/headway#system-requirements

koinedad
Love the idea of freeing up map data
whitten
Is there any support for non-earth maps ? Such as for a DnD campaign or a story set in Middle Earth ? Or even a war game on Earth but with different place names and countries borders names, etc ?
Bedon292
It is all based on OSM data. And I haven't checked in a few months but I believe Maplibre, which is doing the rendering, require the tiles all be in Web Mercator, which is an earth based coordinate system. So you would probably need to hack together something to change the underlying data into what you want. The easiest one would be changing place names and borders on Earth.
clementmas
Is the data shared between vector tiles, geocoding and directions? Or is each feature pulling its data from a different index?
ellenhp
During data preprocessing it all comes from the same OSM extract, except for Pelias which also pulls data in from Who's On First, and Planetiler which also pulls data in from Natural Earth (and a few other sources).

At runtime tiles are served over the wire but everything else happens server-side. So it's a bit of everything :D

karussell
I don't think they share the index as all three parts are different projects.