I'm missing the nuance or perhaps the difference between the first scenario where sending inaccurate time was worse than sending no time, versus the present where they are sending inaccurate time. Sorry if it's obvious.
How do you even get usable microsecond precision sync info from a server thousands of kilometers away? The latency is variable so the information you get can't be verified / will be stale the moment it arrives. I'm quite ignorant on the topic.
When we collected, correlated, and measured all controlling messages in a whole 4G network. Millisecond precision meant guaranteed out of order message flows.
Your speakers do so that people's voices match their mouth movements. The speaker clocks need to be in-sync with the cpu clocks and they operate at different frequencies.
I work at a particle accelerator. We use White Rabbit (https://white-rabbit.web.cern.ch/) to synchronize some very sensitive devices, mostly the RF power systems and related data acquisition systems, down to nanosecond accuracy.
Nitpick: UTC stands for Coordinated Universal Time. The ordering of the letters was chosen to not match the English or the French names so neither language got preference.
I found the most interesting part of the NIST outage post [1] is NIST's special Time Over Fiber (TOF) program [2] that "provides high-precision time transfer by other service arrangements; some direct fiber-optic links were affected and users will be contacted separately."
I've never heard of this! Very cool service, presumably for … quant / HFT / finance firms (maybe for compliance with FINRA Rule 4590 [3])? Telecom providers synchronizing 5G clocks for time-division duplexing [4]? Google/hyperscalers as input to Spanner or other global databases?
Seriously fascinating to me -- who would be a commercial consumer of NIST TOF?
This page: https://tf.nist.gov/tf-cgi/servers.cgi shows that NIST has > 16 NTP servers on IPv4, of those, 5 are in Boulder and were affected by the power failure. The rest were fine.
However, most entities should not be using these top-level servers anyway, so this should have been a problem for exactly nobody.
I believe if you use time.nist.gov it round robins dns requests, so there’s a chance you’d have connected to the Boulder server. So for some people they would have experienced NIST 5 μs off.
Not exactly the topic of discussion but also not not on topic: just wanted to sing praise for chrony which has performed better than the traditional os-native NTP clients in our testing on a myriad of real and virtualized hardware.
Maybe I missed something, but I don't quite understand the video title "NIST's NTP clock was microseconds from disaster". Is there some limit of drift before it's unrecoverable? Can't they just pull the correct time from the other campus if it gets too far off?
27 comments
[ 5.3 ms ] story [ 61.1 ms ] threadSuggestions from the community for more reliable alternatives?
NTP at NIST Boulder Has Lost Power
https://news.ycombinator.com/item?id=46334299
If (and it isn't very conceivable) GPS satellites were to get 5µs out of whack, we would be back to Loran-C levels of accuracy for navigation.
I've never heard of this! Very cool service, presumably for … quant / HFT / finance firms (maybe for compliance with FINRA Rule 4590 [3])? Telecom providers synchronizing 5G clocks for time-division duplexing [4]? Google/hyperscalers as input to Spanner or other global databases?
Seriously fascinating to me -- who would be a commercial consumer of NIST TOF?
[1] https://groups.google.com/a/list.nist.gov/g/internet-time-se...
[2] https://www.nist.gov/pml/time-and-frequency-division/time-se...
[3] https://www.finra.org/rules-guidance/rulebooks/finra-rules/4...
[4] https://www.ericsson.com/en/blog/2019/8/what-you-need-to-kno...
They're also the largest holder of IPv4 space, still. https://bgp.he.net/report/peers#_ipv4addresses
Think Google might have rolled their own clock sources and corrections.
Ex: Sundial, https://www.usenix.org/conference/osdi20/presentation/li-yul... / https://storage.googleapis.com/gweb-research2023-media/pubto... (pdf)
To say NIST was off is clickbait hyperbole.
This page: https://tf.nist.gov/tf-cgi/servers.cgi shows that NIST has > 16 NTP servers on IPv4, of those, 5 are in Boulder and were affected by the power failure. The rest were fine.
However, most entities should not be using these top-level servers anyway, so this should have been a problem for exactly nobody.
IMHO, most applications should use pool.ntp.org
I took too much Adderall today.