HN Explorer

Show HN: Self-Hosted Maps Stack (maps.ellenhp.me)

213 points by ellenhp ↗ HN
Over the past week I built a project to let people self-host an entire maps stack so they don't have to send data to the big G. Right now it includes a base map, geocoder and directions server. Currently only bicycle directions are supported, and I'm only hosting tiles for Seattle because I'm unemployed and can't afford to host data for the whole planet. Check it out!

https://maps.ellenhp.me/

https://github.com/ellenhp/headway

Ultimately I'd like to really focus on transit routing (not implemented) but I'll probably throw a driving mode in there too.

71 comments

[ 5.3 ms ] story [ 114 ms ] thread
ellenhp
I'd really love feedback and suggestions by the way! I have a fair amount of experience in maps but I've never just built an entire maps web app from scratch so I'm kind of stumbling through a lot of it, especially the web frontend stuff.
cmroanirgo
It'd be really great to be able to know what's in the docker black box. It'd be even better to be able to know what the moving parts are for those of us that do self hosting sysadmin style. My VMs are too small for docker... but i'm sure the underlying stack would run without issue.
miyuru
maps.ellenhp.me has only a IPv4 address and it seems to be hosted on digitalocean, which supports IPv6.

would love to have IPv6 support. It should be on default by now, if there aren't any issues hindering support.

tuukkah
For a full multimodal routing engine including realtime data for transit, you should have a look at OpenTripPlanner version 2 (if you haven't already): https://www.opentripplanner.org/

For a single solution to geocoding and autocompletion, there's Pelias: https://pelias.io/

ellenhp
Pelias is pretty difficult to use with the kind of automated config I'm hoping to have for this project but I might give it another go.

OTP is actually really fantastic and it might be a better choice than valhalla's multimodal especially if it can handle GTFS realtime. Thank you for the reminder that it exists!!!

avibryant
This is great! If you're interested, I'd love to have you at a GIS/mapping-focused retreat I'm hosting at https://www.gradientretreat.com/ this summer. You can reach out privately at avi@avibryant.com.
[deleted]
(comment deleted)
mike_d
Email me. I'll give you a server to further development.

All the good open source tile servers have pivoted into commercial businesses, and I'd like to support the development of a project that stays open.

ellenhp
That sounds wonderful! This project is primarily focused on people who want to self-host a google maps replacement, but self-hosting isn't for everyone so I think it would be really cool to host an instance that would work for most people. I'd love for it to grow beyond that though, because lots of people need tiles for static maps or other uses. I will need to re-architect the docker config a bit before it will work at planet scale, because right now mbtiles are bundled with the tileserver image.
colordrops
What about openmaptiles.org?
ellenhp
They don't host anything IIRC. But most of the FOSS maps ecosystem is based around their schema. I know Planetiler uses it at least.
mike_d
They turned into MapTiler, a commercial service.
[deleted]
(comment deleted)
jhgb
One might not need tiles for personal uses, though. If you're not intent on scaling to the level of Google, a tile server seems almost like an overkill. And on some platforms, such as Windows, UMN MapServer is massively easier to install in my experience (not to mention its flexibility).
reducesuffering
I’m working on an app that will extensively use a variety of maps. Did you directly interact with OSM (it’s like a big local DB of map tiles, right?) or did you use any important helper libraries? What are the best resources you used to learn about building city level maps like this?
ellenhp
Lots of it is stuff that I've picked up over the years. I work(ed) in the maps industry for the last 4 years or so, until I was laid off about a week and a half ago. I do interact with OSM a little bit in the docker image generation but the servers (geocoder, tileserver, routing server) all pre-process their data to turn it into a format that's useful for each server's specific purpose. So none of the HTTP requests that my demo server is responding to right now actually directly touch any OSM data.

Most of the OSM data processing is done in the Makefile, but beware that it was very hastily written so it's not a perfect example:

https://github.com/ellenhp/headway/blob/main/Makefile

zylepe
Nice! I was going to point out the project I’ve been working on [1] to reduce the cost but I see you’ve already found it :-) Feel free to reach out it you run into any issues!

[1] https://github.com/onthegomap/planetiler

ellenhp
Planetiler is amazing! Thank you for your work.
hedora
Am I missing something, or does it only cost about $2.27 per month to store all the tiles in Amazon S3? They're definitely not the cheapest game in town.

(Your build example says the tiles are 99GB, and their normal tier is $0.023/GB)

I guess the part of the cost that people actually care about is the bandwidth. How do you suggest dealing with that part of the problem?

ellenhp
Planetiler isn't for hosting tiles, but generating them. For a planet-scale tile generation job it's the difference between hundreds or thousands of dollars in VPS fees and being able to generate them locally on a modestly equipped desktop. Planetiler does drastically reduce cost in that respect.
zylepe
Another option if you want to offload the tiles from embedded in your docker image to static file hosting would be to convert the mbtiles output to pmtiles [1]. Then just upload the pmtiles file to s3 or similar and use the maplibre plug-in to read tiles directly from that large file using byte range requests without a tile server.

There’s a python utility to convert mbtiles to pmtiles, but I’m also planning to add native pmtiles output to planetiler soon [2].

[1] https://github.com/protomaps/PMTiles

[2] https://github.com/onthegomap/planetiler/issues/98

devit
What's the advantage compared to using https://www.openstreetmap.org/ and Organic Maps, or self-hosting them? (the website is https://github.com/openstreetmap/openstreetmap-website and the tile server is https://github.com/gravitystorm/openstreetmap-carto/)
ellenhp
Mostly just that you can self-host this stack and keep your location information private (the way it should be). If you're fine sharing your current and future whereabouts with people then there's no need to self-host so you're free to pick whatever service is the nicest to use. For most people that's going to be Google or Apple maps, but there's definitely a niche for a project like this.

Compared to self-hosting an existing project, it is much easier to bring up a new Headway instance:

`make Seattle.up` will bring up a new instance on 8080.

zaik
OrganicMaps and OsmAnd operate entirely on local map data and don't send your information to remote servers.
ellenhp
Didn't realize that about Organic Maps. It also appears to be open-source which is cool. Is its geocoder any good? Search in OsmAnd is painfully slow and not tolerant of typos at all. Headway uses photon as a geocoder autocompletion system which is incredible and makes the experience miles better than I've ever had with OsmAnd.

I love the idea of offline maps but I've never used an offline maps app with a decent search experience. I also aim to make Headway very transit focused which will allow it to serve real-time data about transit vehicle locations from GTFS realtime feeds.

edit: I just tried out organic maps and it is GOOD. Way better than OsmAnd. I still want to keep working on Headway but this will be useful in the meantime.

999900000999
How much money to add transit directions ?

I would absolutely love if this supported an API to get directions. I've worked on a few projects with public transit directions and I'd love to use a common base instead.

ellenhp
I'm planning on tackling transit this weekend!
999900000999
God Speed, feel free to put up a donation page. I'll definitely donate !
ellenhp
I think this was the push I needed to apply for Github Sponsors. Keep an eye on the project though, I'd really love feedback from people passionate about transit! Feel free to email me too if you have ideas of what the UX should be. ellen.h.poe@gmail.com
3np
In the chance you're open for cryptocurrency donations/support, throw in a BTC, XMR, and/or ETH address and you should see some coming

(Some of us can't or won't use GH sponsors, paypal etc)

FWIW I'm super hyped about transit/walking/biking/driving directions as well.

all2
Minimize left turns? :D
aaronbrethorst
Neat, where are you going to source your data from?
ellenhp
The plan is to use this dataset [1] and do a bounding-box intersection with the chosen city to automatically download appropriate timetables when the user runs `make CityName.up`

[1] https://database.mobilitydata.org/

SlavikCA
Is it possible to use alternative map data source?

I'm looking for a way to generate topographical maps...

Or may be you can suggest any alternative project for that?

hedora
I know you're focused on self-hosting websites on the internet, but I have another, closely related problem.

I have a degoogled Android phone, and location services are one of my bigger frustrations with it. The phone has more than enough room to store the entire planet's maps locally, which means some microg-style replacement for google's location services should be able to compute location name from GPS coordinates, and also pull up map tiles without hitting the network. Both those features are super flaky, especially if the network is down.

This is particularly frustrating because I have Organic Maps installed, and it has already downloaded the necessary databases!

[deleted]
(comment deleted)
KennyBlanken
OSMand can run entirely offline with OpenStreetMap data files. It downloads a base (country-wide-ish), state/region map, and then optionally smaller delta updates published between more significant updates.
tppiotrowski
Hello from a fellow map geek building Shademap[1] in Bothell. Excited to see where your project goes.

[1] https://shademap.app

[deleted]
(comment deleted)
dawnerd
Been wanting to do similar but stopped once I saw just how much data hosting the entire world map takes.
tppiotrowski
Some interesting work is being done to make this affordable (storing a single large planet file in S3 to limit upload costs and using byte-range offset requests to fetch individual tiles) [1]

[1] https://protomaps.com/docs/pmtiles/

tmp538394722
Very nice!

I haven’t looked into the code yet, but I’m especially interested in the routing bit.

As a cyclist I’m almost always disappointed by google, apple, and Valhalla. The cycling directions from these systems are usually better than nothing, but almost always have one or two dangerous or ridiculous segments.

I’ve never tried to solve it myself, only day dreamed.

Any insights?

Doctor_Fegg
You might want to look into https://cycle.travel/ (my site!), https://brouter.de/brouter-web, Komoot or RideWithGPS - all do OpenStreetMap-powered bike directions.

Getting bike routing right is really challenging, partly because often there genuinely isn’t a good route, and partly because standards and infrastructure (and OSM mapping practice!) differ from country to country. I love the challenge and the feedback from users when it works well, but it’s a massive task.

tmp538394722
I just did a quick test of cycle.travel using the last couple trips I took, and it’s pretty good!

It’s anecdotal, but definitely better than what I’m seeing with Val halla / google / apple.

Can you share any details about how your route finding works?

ellenhp
Headway uses Valhalla. Most of what I'm doing with this project is the config to bring up the entire stack easily.
donalhunt
Looks like it uses OpenStreetMap data. Please ensure attribution is displayed as per the license terms.

See https://www.openstreetmap.org/copyright

teh_klev
They already do that.
Freak_NL
It is hidden under the little ⓘ in the bottom right corner. That is a very meagre interpretation of the attribution guidelines. To help combat the idea that a map must be made by some megacorp to be viable, visibility of OpenStreetMap is important.

ellenhp: Why not start with the attribution toggled open, or some alternative way to highlight the map data's provenance?

edit: Attribution is now shown toggled open at page load, resolving this issue.

patriksvensson
This seems resolved. Attribution toggles open when its loaded now.
Freak_NL
Huh, it does. That was impressively fast.
ellenhp
Please open an issue with upstream (maplibre-gl-js) if you're not happy with the attribution that it provides by default. On my end I need to do better to make sure that it's never covered up. I'm really bad at CSS so that's going to be a fun time but obviously needs to be done. There's an issue open to track it now.
donalhunt
Seems to be hit and miss on mobile web (I can trigger the 'i' to show but don't see it in all views).

Currently travelling so will investigate further when somewhere appropriate (may be cached on my phone, something specific to mobile web, etc).

[deleted]
(comment deleted)
jhgb
These things are way more difficult than they should be, really. As evidenced by the effort needed to put this together.
pmontra
I self host OSM by using OSMAnd+ on my phone and downloading the maps of the areas I move to. I'm not particularly interested in directions so I don't know if they work well. It's missing all the satellite and street view layers and they are the main reason I often go to Google Maps.
maxerickson
OSMAnd/OSM data varies a good bit by region. In the US, missing addresses are often the only significant issue with directions (and for OSMAnd, no live traffic data).
aaaaaaaaata
> I'm not particularly interested in directions

What's your use case?

pmontra
I use it build new tracks for cycling and record them. I explore the map (the OSM road type layer and Google satellite view), place markers about 10 meters after the intersections on the road I must take, follow them. I use my old and smaller phone on a phone holder on the handlebar. The screen is almost always turned off. I turn it on when I know I'm about to need to know where to go. I usually remember the places the next time I cycle on the same road.

Navigation when driving my car: not every month, not many times per year. After all I know the roads, I don't need somebody to tell me where to turn. When I do, it's usually the last km, when the chances to miss an intersection are higher. On main roads or close to home, not so much.

franga2000
You can enable the "Microsoft Earth" layer for sattelite imagery in OsmAnd, which is (at least in my region) even better than Google's. I have it set as the "underlay map", which means it's drawn on top of OSM background elements, but below roads, buildings and markings. It also gives you a slider right on the map view to adjust the opacity of the OSM elements so you can see just sattelite imagery when you need it or even set it to somewhere in between.

As for Street View, while I haven't looked into getting it into OsmAnd, Mapillary has an official plugin and while its coverage isn't really comparable, it does often have newer images and often saves me an app switch if I just need to see anything in a general area.

pmontra
Thank you. I didn't know about those layers. Very useful!

I did know about Mapillary. I'll check if its coverage got better.

frzen
Is there any way to write commands where I could give my own time weighting to certain types of roads for route finding?

Google maps, organic maps, osmand+, I still have to double check what crazy suggestion it has made. Single track gravel roads over a perfectly safe alternative a lot of the time.

I have the avoid unpaved roads options ticked... it would be great to have some machine learning magic take a look at a street view picture and decide if there is grass growing up the middle of the road and add a weighted chance of needing to reverse 200 metres when there is other traffic

uptime
Headway is a great name for wayfinding! Looking forward to this evolving.