Show HN: Building a GPS receiver (axleos.com)
Shortly after publishing my iOS 4 jailbreak last October[1], I got to work on my next hobby project: a from-scratch homebrew GPS receiver, which can solve the user’s location solely from billions of radio antenna samples.
I took a commodity SDR (alongside the Python standard library and numpy) and built a signal processing pipeline that can detect and track GPS satellites over many minutes, drop and pick up satellites as they come in and out of view, and precisely determine the user’s position and clock inaccuracy.
All told, gypsum can go from a cold start to a fix on the user’s position, and the precise time, in less than a minute of listening to the antenna. I went on a journey of learning how to detect and track satellite signals that are literally too quiet to hear, and I hope that some of the magic comes through in the posts!
After implementing this myself and walking the long road of getting it working, I’m left completely stunned by the brilliance of GPS, across so many axes. I hope you enjoy the read!
On a more personal note, I’ll be starting a new job next week which isn’t as amenable to publishing side projects, and therefore this will be my last publicly-published project for some time. I’ve had great experiences making and sharing projects on here, and I’m really grateful for the positive feedback that’s been shared!
134 comments
[ 6.3 ms ] story [ 211 ms ] threadWhenever I see "from scratch," I'm always curious to see how from scratch the author actually means so I'll admit I was a bit disappointed to see that the hardware was just RTL-SDR. Still, the protocol decoding was very interesting and the result is great.
> GPS was launched in 1978, which was 45 years ago at time of writing. Five billion people are currently under 40 years old, so well over half the world’s population has never existed in an environment but this.
A note based on this. While GPS was around since 1978 the signal was intentionally degraded with a process known as "selective availability" until 2000. This largely rendered GPS unusable for many many purposes, definitely useless for road navigation, it had some limited utility in areas like backcountry navigation and was definitely useful for marine navigation.
> gypsum can go from a cold start to a fix on the user’s position, and the precise time, in less than a minute of listening to the antenna
This is very impressive and outclasses what I see even commercial receivers doing today, do you have any idea how? I remember on road trips in the early 2000s I would have to sit on the side of the road and wait for the GPS receiver to get a fix (a 15-20 minute process, when it worked) before we could leave. Or, more likely, my mother would just start driving with paper maps.
That's cool, I can understand just enough about what's going on there to know I have absolutely no idea what's going on for 90% of that article. Excellent to learn from.
Re. slow TTFF, or time-to-first-(position)-fix on older hardware, this essentially stems from advancements in processing power.
Traditionally, GPS receivers would need to download the ‘almanac’ of all the satellites, which takes a minimum of 12.5 minutes (under certain conditions) due to the GPS data transmission format and speed. With modern processing power, though, receivers (including gypsum) can just ‘brute force’ the search space to find the in-view satellites, instead of using the hints downloaded over the air. This is the technique described at the end of Part 1.
The 12.5 minutes includes a rough multi-week almanac which you could perhaps brute force given available compute and receive capability (original GPS receivers have a single channel receiver and minute compute capability) but they more importantly include the ephemerides, precise data about exactly where the birds are and the atmospheric conditions, replaced hourly by a ground station. You can't "brute force" these - they're parameters measured by someone with objective truth like "I, a massive NASA satellite ground terminal in Florida, am definitely not moving, therefore this GPS bird #14 is 0.08 metres away from where it should be, I will adjust the data for the next hour accordingly".
https://en.wikipedia.org/wiki/GNSS_augmentation
In this case, I was meaning to refer to brute-forcing the Doppler-shifts and PRN phases of each satellite, not the orbital parameters themselves. The project in the OP is able to get a position fix in less than a minute because, if the subframe timings are convenient, you can retrieve the necessary ephemeris parameters from the subframes in that span (and down to as little as 18 seconds in ideal conditions, if my back-of-the-napkin is right).
Most decent modern-ish receivers tend to have pretty speedy aquisition time without any assistance data. For example, the reasonably ancient GPS running watch I use can usually get a GPS lock in a couple of minutes from cold with no internet access (in a wrist-sized device running on battery!), and even the two decade old SiRFstarIII chipset is specced to have a sub-minute cold start time without assistance and much shorter with - though I think that chipset was pretty advanced for the time.
But it's very much feasible to 'brute force' your initial signal lock by searching for all gold codes at a range of frequency offsets.
And it doesn't take 12.5 minutes to get the ephemerides - the almanac is sent in paginated form which is why it takes so long, the ephemerides are sent more often - they repeat every 30 seconds, and they're enough for a navigation fix.
Although 30 seconds isn't amazing, so cell phones do use their data connection to shortcut that wait.
I have no idea, but the claim was that you get faster fix with brute force when I know that's not why it's fast in practice.
> But it's very much feasible to 'brute force' your initial signal lock by searching for all gold codes at a range of frequency offsets.
I hadn't even imagined this constituting "brute force". Is my phone using "brute force" to find the WiFi router? At some point it's not really "Brute force" it's "There are a handful of options, try all of them" and GPS seems past that point especially on modern hardware.
This actually reminded me of a (possibly no longer extant) design choice in Encrypted Client Hello - we don't necessarily know if the encryption was done with key F we gave out yesterday afternoon or key G which we just began using an hour ago, do we need a way to signal that in the connection? No, just try all valid keys. If you can't afford to try more than two keys, make sure you only roll them slowly so you won't need to.
Your phone only needs to listen to the WiFi router on one channel at a time in operation, and the signal parameters are well enough defined that they can be scanned quickly. A GPS receiver requires at least 4 parallel channels to achieve a position solution, and there are up to 32 possible codes the satellites could be at. Scanning 6 channels across 32 codes, and then also sweeping phase and doppler shift to lock them , just to 'discover' if there is a valid signal there takes time, and this is what older receivers had to do. Modern receivers tend to just 'brute force' this by having an entire receive pipeline dedicated to every possible PRN all the time, and possibly even correlate multiple doppler shifts simultaneously as well, so they effectively have 32 (or more) receive channels, despite only ever expecting a maximum of 12 birds being visible. The extra channels are necessary more or less exclusively to reduce acquisition time, so I think it's fair to call them 'brute force'.
Early receivers were a lot less advanced than modern receivers - one of the key functions of the almanac is to help receivers figure out what satellites they can expect to see - thus greatly reducing the range of gold codes and time offsets they have to check.
Unlike wifi, GPS signals are below the noise floor until the gold code is applied to despread the signal, and the gold code has to be synchronized with the received signal to detect it.
The gold codes are pseudorandom and designed to stop signals interfering with one another. Unless you know which gold code you're looking for, and find its time offset (accurate to about 2 chips in 1023) you can't tell it apart from noise.
You also don't quite know the frequency you're looking for - partly due to the imprecision of the receiver clock, partly because GPS satellites move very fast and so can have a lot of Doppler shift (depending on where they are in the sky relative to the receiver of course)
Back when receivers had more limited physical hardware, searching through ~30 different satellites, multiplied by ~500 different gold code offsets, multiplied by a few different Doppler shifts could be a slow process. Especially if you'd found a handful of satellites, so some of your receiver channels were tied up with tracking leaving you with fewer for searching!
So ignoring the almanac and brute forcing every satellite, gold code offset and doppler shift is one of the many ways performance has increased since this stuff was developed in the late 1970s.
Ah, the almanac part I completely forgot about, that makes a lot of sense, I read that part but forgot how it USED to be done when we couldn't just throw cycles at it.
Somewhat surprisingly, if you went back 15-20 years, a lot of what the author is doing in software here would have been done in hardware.
GPS receivers used to market themselves by the number of tracking channels they had, as cheaper receivers might only have the hardware needed to track 6-8 satellites while a more expensive receiver might track 12.
So this software-defined receiver actually implements quite a bit of what would otherwise be hardware. And of course it can track every satellite in view.
The software-defined approach has some powerful benefits - for example, initial satellite acquisition involves calculating cross-correlation between the received signal and various gold codes. Being able to do this in the fourier domain lets you acquire signals pretty fast!
If you want a hardcore DIY GPS receiver, going right down to the transistor level, you'd probably enjoy reading https://lea.hamradio.si/~s53mv/navsats/theory.html - an 1990s era DIY GPS receiver, complete with hand-drawn schematics, hand-drawn PCBs, even a hand-made antenna.
― Carl Sagan, Cosmos
Ever since I've seen the project by Andrew Holme (mentioned in sibling comments) years ago it has been on my wish list to replicate, but analog/RF signals are dark magic to me.
Now I feel like I can skip the hard RF frontend bit and play with the software by using the SDR I already have.
https://en.wikipedia.org/wiki/Automotive_navigation_system. Moreover, while often not ideal in dense urban environments(modern receivers often struggle here anyways), by the late 90s differential GPS augmentation was available in cars as well, which was available in dense coastal population areas like NYC. Old auto nav systems were clunky and with overall shitty map data but they weren't "definitely useless" due to SA.
EDIT: I'll concede they were pretty bad, but SA was only one factor. With today's computing power and higher quality maps you could more easily adapt to the SA position error if it were an issue as well.
Automotive GPS systems existed pre-2000. So did dead-reckoning systems. Did people use them at the time? Some did. It was an amazing technology compared to the alternative, which was manually navigating a paper map.
But you'd often get errors large enough (50m avg) that it wouldn't accurately identify your location on roads close enough to provide accurate instructions. If you gave any of that tech to someone today to use, they'd think it was broken.
[1] https://www.bosch-presse.de/pressportal/de/en/navigation-sys...
I would like to point out the insanely good design of the GPS radio layer (the L1+L2 signals).
Even 46 years on, the radio layer is fully forwards and backwards compatible, and a bunch of important metrics like time to first fix and user equivalent range errors have both improved by factors of 10-1000, with no incompatible change needed to the protocol.
The total RF transmit power to provide service to the whole earth is less than the electricity consumption of a typical US house (far less than 5G or TV or AM/FM radio), and well below the noise floor. That's possible due to clever use of stacked gold codes.
The design has allowed frequency-sharing with competing systems (eg. Galileo) - you don't see mobile phone networks doing that!
The actual signal sent has allowed things like carrier phase decoding, due to the locking of the phase between the modulated data and the carrier, which in turn gives far better pseudoranges and accuracy.
Overall, the designers either had incredible forethought, or incredible luck, or some combination of the two.
I would disagree strongly with this. I took a roadtrip in 1999 using a Delorme Earthmate Hyperformance GPS receiver, the RS-232 version, plugged into a Toughbook running Delorme Street Atlas USA, I believe it was version 6.0.
It provided perfectly usable directions all the way across the country. It didn't do lane guidance (which I don't find terribly helpful anyway), but some time in advance of every turn, it would announce the turn, including the street name.
That version even had voice recognition, so you could say things like "are we there yet?" and it would announce the ETA to both the next stop and the final destination, along with current location. Lots of fun!
30 meters (typical worst-case CEP under SA) is plenty accurate for road navigation in all but the densest areas, and even then, just glance at the map. Once you're out on the open road, it's brilliant. Rock out to some mp3's until the voice pipes up with the next maneuver.
I have one of these, but don’t really know how useful it is or what I can do with it, but like all of my random antique hardware, I don’t really have any good reason to part with it either.
Do you have any ideas for what to do with it?
For that matter, what is a good GPS unit in modern times?
These days, it depends on what you're doing with it, but it's hard to go wrong with this thing that claims to have a U-Blox 8-series chipset:
https://www.aliexpress.com/item/32816656706.html
Uniquely among such cheap receivers, that unit can even be configured to report raw data, and is thus the darling of the Galmon project for low-cost observers starting up. Also fun if you want to play with (single-band) RTK, etc.
(Dual-band has gotten "affordable", by which I mean "under a kilodollar", which is tremendous given where it was just a few years ago, but you'd probably know if you had a use for dual-band.)
GPS receivers will look for the 'strongest' PRN signal in the noise, so broadcasting louder than the (incredibly weak!) C/A signal is a valid way to jam or spoof GPS. It is, however, generally illegal for civilians.
GPS receivers operating with good practice do tend to try to mitigate this sort of attack, by (for example) ignoring signals with a too-high power level. It's a bit of a cat and mouse game, and there are academic papers exploring each side.
Lastly, GPS receivers also need to deal with interference from GPS itself! If GPS signals bounce off surfaces before reaching the receiver, the receiver might see two sets of GPS signals: one that arrived directly, and one that was scattered off a surface and arrives a bit later. This is called ‘multipath interference’, and part of what goes into making GPS receivers work well is mitigating multipath interference.
I don't think this is actually the case. In a spoofing scenario, all of the rogue signals would typically be generated by a single terrestrial station. The time of flight of all of the generated signals will be the same, so all that matters is the position solution reflected in the transmitted signals, as the fundamental principle of GPS based on TOF is no longer in play. So I'd think that in a typical spoofing scenario, all receivers thinking they're in more or less an identical location is what you'd expect.
It might be possible in a borderline case for the receiver to receive some spoofed signals and some real signals simultaneously, in which case you'd expect weird results, but I think you'd definitely see a correlation around the position being broadcast by the spoofer.
You can blast out fake data, but depending on what you mean by "large area" and a "point", I don't think what youre suggesting is possible. To trick GPS receivers you end up broadcasting fake signals from multiple GPS satellites, so receivers in different areas will be processing it differently and come up with different coordinates.
Also see this one: https://www.lloydslist.com/LL1148748/War-zone-GPS-jamming-se...
I think it was due to Israel's some kind of defense measure against Iran.
There's a report on very similar jamming happening during the Syria conflict that will hopefully be enlightening as the methods and actors are presumably similar https://c4ads.org/wp-content/uploads/2022/05/AboveUsOnlyStar...
This may be theoretically possible but is, in reality, practically impossible.
Embedded within the GPS signal is the ephemeris data which, among other things, includes each satellite's location in space.
Receivers calculate position by calculating the difference between the time a signal was received and the time stamp encoded in the signal itself.
By analyzing the signals from a minimum of four satellites (one for each dimension in time and space), a receiver calculates where it is.
To spoof all phones on Earth, you would need to trick each receiver individually. Since receivers are passive, they don't identify themselves, and there would be no way to target each individual receiver, making them think they're somewhere they're not.
1. Jamming is obfuscating a signal, usually by creating a lot of noise that makes the real signal hard to find. Spoofing is impersonating a signal.
Not necessarily jamming.
Even if a couple of these signals are degraded, wrong, or missing, most phones will come up with a relatively accurate location using the remaining data.
https://youtu.be/Z5N4CqJLAhQ?si=lvaQZv-WG3ab_gEI
I didn't finish reading the whole thing but was curious. Is there any way of brute forcing it or some other trick to get the chipping sequence to get the P code for more precise GPS?
For those interested in the story of the development of GPS, I found “GPS Declassified” by Richard Easton to be an engaging retelling.
https://en.wikipedia.org/wiki/Hedy_Lamarr
* https://www.youtube.com/playlist?list=PLGvhNIiu1ubyEOJga50LJ...
This was true, but not any more. You can get truly impressive “direct RF sampling” or “direct RF conversion” receivers that are more than fast enough for GPS. For example:
Xilinx RFSoc: https://www.mouser.com/datasheet/2/903/ds889_zynq_usp_rfsoc_...
A nice National Instruments article: https://www.ni.com/en/solutions/aerospace-defense/radar-elec...
And their referenced off-the-shelf hardware: https://www.ni.com/en-us/shop/category/flexrio-custom-instru...
One might be forgiven for being a bit puzzled as to why NI thinks that direct RF conversion is cost-effective but nonetheless sells the device for $30k :) That being said, if I were prototyping a system that wanted phase-coherent wideband reception around 3 GHz and I had a proper lab and budget, I’d buy a few of these. If I were to go to production, I’d either wait for costs of a homemade board to come down a bit or see whether a traditional heterodyne receiver could do the trick.
Hmm. For military applications, if I were concerned about really advanced RF-seeking weapons pointed at me, a direct conversion receiver is probably great — there won’t be any leakage of the LO that an enemy device could try to detect.
Why would an LO be more of an issue than your sample clock?
edit: missing word
In todays world with everything software reconfigurable, changing the sampling rate or local oscillator frequency is very do-able.
- GPS works even in airplane mode (while on a literal airplane) - It works without cell service, or wifi, or anything - The United States of America controls the GPS constellation, and they can (and have!) turned off GPS off certain regions at will when necessary (which has prompted other countries to launch their own GNSS constellations) - GPS satellites don't send down a location, they only send down time
I think it's a really fun exercise to do this with data you receive on your phone. Your phone has a direct link to satellite.
(side note: I recently learned the basic principles of star navigation, and while it is a completely different mechanism, it also relies very much on keeping accurate time, which I thought was a fun symmetry!)
Since A-GPS uses the cell tower to get the list of satellites in view, the GPS on some cellphones will keep working when cell service is lost but won't start working if cell service is unavailable.
I think this means my Samsung doesn't actually have GPS, since fallback to unassisted GPS has never worked for me (yes, I've tried waiting far longer than 15 minutes).
Maybe you can excuse a mental model that doesn't make the GPS vs A-GPS distinction, since A-GPS is often sold as GPS.
[0]https://en.wikipedia.org/wiki/Assisted_GNSS [1]https://news.ycombinator.com/item?id=40042686
Usually this just means that it has a weak antenna or catches too much noise, which isn't as much of an issue when assisted. It will probably work just fine unassisted under perfect conditions, but struggles otherwise. Apparently it's pretty common for modern phones, but there are exceptions.
Fancier software would sit there forever trying to get lock, but more primitive one's could get a lock and after that the fancier ones could as well.
This meant I got to "boot" the lock by hamgps first and then I could switch to maps and other software.
Functionally on most devices losing network coverage renders GPS useless. I keep telling people to download OsmAnd if they want to be able to view maps on a plane or get home from their hike outside cell range. Google maps will try to cache maps to some degree nowadays but it tends to be very flakey and it seems to be very easy to accidentally get it to drop its cache when you're outside cell coverage.
All those things are neat, but mostly what I've learned is that quite a lot of people, otherwise apparently reasonably smart and competent and toting a whole stack of prestigious degrees, have ghastly flaws in their mental model of what GPS is.
(and contrary to popular belief, I think the average person is interested in understanding this, they're more motivated to understand things that actually matter to them, that are in their hands every day. A lot just have an emotional aversion to math due to bad school experiences, but they are genuinely curious)
Maybe this is a conception that some people have when their first experiences of using GPS was on a smartphone?
But my first couple GPS receivers were standalone devices without any sort of data connection, so it seems obvious to me that GPS doesn't require data.
My only exposure to GPS without data came from outdoor sports where GPS watches and PLBs are common, as are offline mapping applications. I don't think it's obvious to most casual users of GPS technology today that it is entirely independent from data connectivity.
The GPS almanac data they transmit is effectively location. It's not literally location, but the P code isn't literally time either.
> and they can (and have!) turned off GPS off certain regions at will when necessary
As I understand it, those capabilities are no longer present in newer (possibly all active?) GPS satellites.
They do send a location - to be more exact, they send rough locations of all satellites in the constellation (almanac) and precise location of themselves (ephemeris). Devices like mobile phones, however, usually get that data from other sources because it's much faster than listening to data over GPS.
Heh, but they aren't the only global positioning system out there (nor the most accurate). EU, China, India, and Russia all have their own.
While we're at SDRs, ITAR is also responsible for takedown of passive radar GNU Radio module made by Kraken RF team.
[1] https://www.space.commerce.gov/itar-controls-on-gps-gnss-rec...
https://hackaday.com/2022/11/19/open-source-passive-radar-ta...
[1] https://www.rtl-sdr.com/sdrdue-updated-passive-radar-softwar...
Comment thread from 2023-02-10:
> We are attempting to clarify if it is legal for us (KrakenRF, a US company that provides a physical SDR product) to also provide our own open source software that is made by us. As that could be seen as providing a full PR system.
Is the latest I found from them.
Almost any lawyer won't present the world as black and white, but rather in quantities of risk - and even saying "we've taken the project down and it wont be coming back" is a risk if that attracts attention to your past distribution of the software and causes others to mirror it from archives.
Even that’s asking for a lot.
Qualities of risk is more likely.
Additionally, it seemed like this page[1] (discussed previously[2]) details some reasons why some publicly visible source code projects should be able to include code that implements things otherwise under export control.
[1] https://www.unr.edu/sponsored-projects/compliance/export-con...
[2] https://news.ycombinator.com/item?id=40041198
https://github.com/barbeau/gpstest
I've been fascinated for years how badly GPS does altitude (mean-sea-level)
in the USA they had to build an augmentation system for airplanes for altitude (WAAS)
It just so happens I've got an RTL-SDR, a GPS receiver that outputs raw pseudoranges, and a signal splitter that lets me put the signal from one antenna into two receivers.
So if you like I can get the pseudoranges out of a commercial GPS receiver, and the raw signal from an RTL-SDR at the same time, which might help you pinpoint your last bit of location inaccuracy.
Would you be interested in that? Or do you consider this project complete?
I noticed the opposite, it always fails to locate any satellites, even when GPS is still turned on in aeroplane mode. I'm not sure why.
Yes, and I use that to take pictures of features below my as we fly from one place to the other.
If you have a iPhone, when you land, those pictures will be associated with the place you were when the photo was shot. This enables you to locate those curious features you happen to see.
What does a job have to do with publishing hobby projects?
(Other jobs may just take up a lot of energy and/or time.)
Where and how is that legal? You're supposed to be an employee, not a slave.
And what they can claim varies but can go quite far. For example, even employee-friendly California with a law specifically to limit far reaching clauses (https://law.justia.com/codes/california/2011/lab/division-3/...) allows terms in employment agreement that assign to the employer the ownership of "invention[s] that the employee developed entirely on his or her own time without using the employers equipment, supplies, facilities, or trade secret information" as long as they "Relate ... to the employers business, or actual or demonstrably anticipated research or development of the employer". For a megacorp, that can be basically everything.
The second part is that in some countries you can be fired for any reason, or no reason, except for specific protected reasons. "We didn't like your blog post and are worried that you might accidentally reveal company secrets in one of them in the future, so we're letting you go" would be legal in many places. "All social media or personal web site content has to be reviewed to make sure you're not leaking company secrets" is also a demand some companies make.
Likewise "we want you to give 100% at work and your side project clearly shows you're spending too much time elsewhere so we don't believe you're working to your full potential", even though stupid, would be a perfectly legal justification to fire someone in many countries. Or just the manager silently thinking that and silently retaliating through e.g. denying promotions.
I didn't finish reading the whole thing but was curious. Is there any way of brute forcing it or some other trick to get the chipping sequence to get the P code for more precise GPS?
If you know whats the P code roughly is, which you can know from L1CA code and handover words, you can try using both possible W code value (its 0 or 1) and that W code practically dissappear, until the W code move to its next iteration and you need to guess again. This strategy is called semi-codeless tracking.
People did this so they can access the signals at L2 frequency (1227.6 MHz), which because its at a different frequency than L1 (1575.42 MHz), they got slowed down by different amount through the ionosphere. So by comparing the signals together, you can remove the ionospheric effects and get more accurate position.
These days you don't need all of this, GPS now have public signals at L2 and now smartphones have these fancy dual-frequency receivers