GPS timing modules should have a sawtooth correction value that will tell you the error between the upcoming pulse and GPS time. The issue is that PPS pulse has to be aligned to the receiver's clock. Using that will remove the main source of jitter.
My experience with rt Linux is that it can be exceptionally good at keeping time, if you give up the rest of the multitasking micro sleeping architecture. What do you need this accurate time for? I'm equally sure, as acknowledged, the multipath routing isn't helping either.
Synchronising the clocks on network connected audio devices (ADCs, DACs, DSP processors) on a LAN (https://en.wikipedia.org/wiki/Audio_Video_Bridging), or over the internet (broadcast-grade live streaming). This, and related standards, are more or less the norm in live sound and high-channel-count digital recording setups.
When it comes to realtime guarantees, bare metal code on a dedicated IC or MCU is by definition better than anything running on a general purpose OS on a general purpose CPU. Even if you tune the hell out of it, the latter will have more room for bugs, edge cases, drift, delayed powerup, supply chain idiosyncracies, etc. FYI GNSS processing chips cost $1.30 these days. https://www.lcsc.com/datasheet/C3037611.pdf
GPS modules need to be put in a special stationary mode (and ideally measured-in to a location for a day or two) to get accurate timing. I'm consistently achieving ca. 10ns of deviation. Hope the author didn't forget this. (But it might also just be crappy GPS modules, I'm using u-blox M8T which are specifically intended for timing.)
The problem with these types of posts is that this is an area that many are unfamiliar with (at least not in depth) and making lots of authoritative statements makes it believable at face value. There are so many variables to network time sync that you design to minimise them. For example no multipath and no asymmetry unless you have PTP P2P transparent clocks everywhere.
The author also mixes precision with accuracy and relies on self-reported figures from NTP (chrony says xxx ns jitter). With every media speed change you get asymmetry which affects accuracy (not always precision though). So your 100m->1G link for example will already introduce over 1 us of error (to accuracy!), but NTP will never show you this and nothing will unless you measure both ends with 1PPS, and the only way around it is PTP BC or TC. There is a very long list of similar clarifications that can be made. For example nothing is mentioned about message rates / intervals, which are crucial for improving the numbers of samples filters work with - and the fact that ptp4l and especially phc2sys aren't great with filtering. Finally getting time into the OS clock, unless you use PCIE PTM which practically limits you to newer Intel CPUs and newer Intel NICs, relies on a PCIE transaction with unknown delays and asymmetries, and without PTM (excluding few NICs) your OS clock is nearly always 500+ ns away from the PHC and you don't know by how much and you can't measure it. It's just a complex topic and requires an end to end, leave no stone unturned, semi-scientific approach to really present things correctly.
They state there is a problem but then state they are happy with what Chrony is doing so what exactly is the problem they are trying to solve? What on their network is requiring better than 200ns? or even 400ns for that matter? Not in theory but in reality? Also there are optimizations they are missing in this document [1] such as disabling EEE.
On a more taboo note, while RasPi's can be great little time servers they have more drift and will have higher jitter but that should not matter for a home setup and should not be surprising. If jitter is their concern then they should consider using mini-pc's, disable cpuspeed, all power management and confine/lock the min/max speed to half the CPU capabilities and disable all services other than chrony. It will use more power but would address their concerns. They could also try different models of layer 2 switches. Consumer switches will add some artificial jitter and that varies wildly from make, model and even batch but again for a home network that should not matter. I think they are nitpicking. Perfect is the enemy of good, especially in a day and age when people prefer power saving over accuracy.
[Edit] As a side note the aggressive min/max poll settings they are using can amplify the inefficiencies of consumer switches and NICs regardless of filter settings and that can make the graphs more chaotic. They should consider re-testing that on data-center class servers, server NICs and enterprise class switches or just reduce the polling to something reasonable for a home network minpoll 1 maxpoll 2 for client, minpoll 5 maxpoll 7 for edge talking to a dozen stratum 1's with a high combinelimit. Presend should not be required even with default ARP neighbor GC times and intervals. Oh and if you want to try something fun with the graphs, run chronyc makestep ever minute in cron on every node. yeah yeah I know why one would not do that and its just cheating.
The paths on the network with the MLAG is NOT likely the issue. There is a serialization delay different between the NICs in the NTP servers and switches. 100M takes more time then 1G which takes more time then 10G which takes more time then 40G. Also the UI kit is store and forward (not sure on the 10G but the desktop one is) switching and the Arista kit is cut thought (at around ~500 byte packets IIRC). The end to end paths are not the same given the link speed difference and that is the source of the variations. The MLAG hashing is in hardware and will not have an effect, also IIRC you can set the LAG hash to be SRC/DST on both L2 and L3 even on a L2 link.
18 comments
[ 3.3 ms ] story [ 36.0 ms ] thread"A man with a watch knows what time it is. A man with two watches is never sure."
https://en.m.wikipedia.org/wiki/Segal's_law
Synchronising the clocks on network connected audio devices (ADCs, DACs, DSP processors) on a LAN (https://en.wikipedia.org/wiki/Audio_Video_Bridging), or over the internet (broadcast-grade live streaming). This, and related standards, are more or less the norm in live sound and high-channel-count digital recording setups.
Linux PTP (https://linuxptp.sourceforge.net/) and hardware timestamping in the network card will get you in the sub 100ns range
Graham: Synchronizing Clocks by Leveraging Local Clock Properties (2022) [pdf] (usenix.org) https://news.ycombinator.com/item?id=44860832
In particular the podcast about Jane Street's NTP setup was discussed.
The author also mixes precision with accuracy and relies on self-reported figures from NTP (chrony says xxx ns jitter). With every media speed change you get asymmetry which affects accuracy (not always precision though). So your 100m->1G link for example will already introduce over 1 us of error (to accuracy!), but NTP will never show you this and nothing will unless you measure both ends with 1PPS, and the only way around it is PTP BC or TC. There is a very long list of similar clarifications that can be made. For example nothing is mentioned about message rates / intervals, which are crucial for improving the numbers of samples filters work with - and the fact that ptp4l and especially phc2sys aren't great with filtering. Finally getting time into the OS clock, unless you use PCIE PTM which practically limits you to newer Intel CPUs and newer Intel NICs, relies on a PCIE transaction with unknown delays and asymmetries, and without PTM (excluding few NICs) your OS clock is nearly always 500+ ns away from the PHC and you don't know by how much and you can't measure it. It's just a complex topic and requires an end to end, leave no stone unturned, semi-scientific approach to really present things correctly.
Meanwhile, here's some other articles:
NTP: https://austinsnerdythings.com/2025/02/14/revisiting-microse...
PTP: https://austinsnerdythings.com/2025/02/18/nanosecond-accurat...
https://www.jeffgeerling.com/blog/2025/diy-ptp-grandmaster-c...
On a more taboo note, while RasPi's can be great little time servers they have more drift and will have higher jitter but that should not matter for a home setup and should not be surprising. If jitter is their concern then they should consider using mini-pc's, disable cpuspeed, all power management and confine/lock the min/max speed to half the CPU capabilities and disable all services other than chrony. It will use more power but would address their concerns. They could also try different models of layer 2 switches. Consumer switches will add some artificial jitter and that varies wildly from make, model and even batch but again for a home network that should not matter. I think they are nitpicking. Perfect is the enemy of good, especially in a day and age when people prefer power saving over accuracy.
[Edit] As a side note the aggressive min/max poll settings they are using can amplify the inefficiencies of consumer switches and NICs regardless of filter settings and that can make the graphs more chaotic. They should consider re-testing that on data-center class servers, server NICs and enterprise class switches or just reduce the polling to something reasonable for a home network minpoll 1 maxpoll 2 for client, minpoll 5 maxpoll 7 for edge talking to a dozen stratum 1's with a high combinelimit. Presend should not be required even with default ARP neighbor GC times and intervals. Oh and if you want to try something fun with the graphs, run chronyc makestep ever minute in cron on every node. yeah yeah I know why one would not do that and its just cheating.
[1] - https://chrony-project.org/examples.html