33 comments

[ 3.6 ms ] story [ 61.5 ms ] thread
Well, where did NTP at NIST last put it? Did they look there?
Man, they're having a hell of a time up in Boulder.
NIST campus status: Due to elevated fire risk and a power outage for the Boulder area, the DOC Boulder Labs campus is CLOSED on December 19 for onsite business and no public access is permitted; previously approved accesses are revoked.[1]

WWV still seems to be up, including voice phone access.

NIST Boulder has a recorded phone number for site status, and it says that as of December 20, the site is closed with no access.

NIST's main web site says they put status info on various social media accounts, but there's no announcement about this.

[1] https://www.nist.gov/campus-status

Can anybody expand on the implications of this?

Being unfamiliar with it, it's hard to tell if this is a minor blip that happens all the time, or if it's potentially a major issue that could cause cascading errors equal to the hype of Y2K.

This was an NTP 0 server right? What is the actual failback mechanism when that level of NTP server fails?

This is some level of eldritch magic that I am aware of, but not familiar with but am interested in learning.

Wind gusts were reaching 125 MPH in Boulder county, if anyone’s curious. A lot of power was shut off preemptively to prevent downed power lines from starting wildfires. Energy providers gave warning to locals in advance. Shame that NIST’s backup generator failed, though.
> Wind gusts were reaching 125 MPH in Boulder county, if anyone’s curious.

That's some strong winds! What's causing such strong sustained/gusty winds that long? I'm hearing about this weather phenomenon for the first time.

> Facility operators anticipated needing to shutdown the heat-exchange infrastructure providing air cooling to many parts of the building, including some internal networking closets. As a result, many of these too were preemptively shutdown with the result that our group lacks much of the monitoring and control capabilities we ordinarily have

Having a parallel low bandwidth, low power, low waste heat network infrastructure for this suddenly seems useful.

So far I think I'm still seeing one of them in my peers list for my public-ish NTP server:

         remote           refid      st t when poll reach   delay   offset  jitter
    ==============================================================================
    +time-e-b.nist.g .NIST.           1 u  372 1024  377  125.260    1.314   0.280
This makes me wonder, if you take the average time of all wristwatches on the planet, accounting for timezones and throwing out outliers, how close would you get to NTP time?

And how many randomly chosen wristwatches would you need to get anything reasonable?

For future reference of civilization: if a facility is critical, it must have a SMR.
It’d be a good idea to protect our infrastructure from the climate we created.

It’s just a good idea, though, not a greedy one… so it won’t happen.

One question I have is did DOGE decisions have anything to do with this? Because I know they took knives to NIST.
This is terrible. It’s always sad to hear about things like this.
Can anybody speak to the current best practices around running underground power lines? I see these types of articles about above-ground distribution systems from time-to-time, particularly in California. I feel lucky that my area has underground power, but that was installed back in the 1980s. Would it be prohibitively expensive for Boulder’s utility provider to move to underground distribution? I can’t help but think it could be worth the cost to reduce wildfire risk and offer more reliable service.
My local GPS + PPS based NTP server finally pays off.
(comment deleted)
We had some fun requesting key for accessing nist time servers. the process is (quoted from website)

NIST currently offers this service free of charge. We require written requests to arrive by U.S. mail or fax containing:

Your organization’s name, physical address, fax number (if desired as a reply method).

One or more point-of-contact personnel or system operators authorized to receive key data and other correspondence: names, phone numbers, email addresses. Up to four static IPv4 network addresses under the user’s control which will be allowed to use the unique key. By special arrangement, additional addresses or address ranges may be requested.

Desired hash function (“key type”). NIST currently supports MD5, SHA1, SHA256, and HMAC-SHA256. Please list any limitations your client software places on key values, if known: maximum length, characters used, or whether hexadecimal key representations are required. If you prefer, please share details about your client software or NTP appliance so we can anticipate key format issues. Desired method for NIST’s reply: U.S. mail, fax, or a secure download service operated by Department of Commerce.

NIST will not use email for sending key data.

ps. there actually seems to be improvement over what they had year ago. they added "secure download service". and previously they had message that nobody assigned to actively monitor mailbox so if you didn't get key, please email us so we will check it

Generator maintenance is a thing.
It’s fine. The public pays for sci-fi clocks used by NIST and the Navy and we get shit latency over NTP and a WWVB signal that barely reaches a huge chunk of the country. CLOCKS WE PAID FOR. Jane Street gets lightning access to clocks and our pension managers get their NTP trades front run. NTP is a disgrace and an insult when it is working.
The rest of the snark/ bitterness aside ... NTP takes latency as a constraint, and accommodates for it.

And far from a disgrace, NTP was a brilliant design for its time, and has proven to be far more future-proof than should have been expected.

“snark bitterness side” I see so I’m not entitled to my feelings and I’m holding it wrong. No. It’s a disgrace. The latency is shit the precision is shit and WWVB to go with it is an insult.

Future proof? Are you living in the same future I am where our trades are front run? I notice you didn’t engage with the substance of my attack at all.