85 comments

[ 2.3 ms ] story [ 248 ms ] thread
Wait, so all of the hand wringing over the crappy UI was pointless because it was actually a case of miscommunication?
I wouldn't call it hand wringing, as I think all the points were valid, but yes, the employee sent out the message he intended to (the real alert over the test alert).
No, according to the end of the posted article, poor UI and a lack of necessary safeguards were part of the problem, just not the entire problem.
I don't really understand how it is part of the problem. If the worker really believed an attack was incoming, confirmation screens wouldn't help at all.
I guess it would depend on how that confirmation screen worked. If the design asked for a confirmation from a second user, then a single user's confusion wouldn't have resulted in the error.
Not only that, but once they realized the message was sent in error they didn't have a simple way to correct course so there was a big delay between the realization and informing the terrified public.

Your suggestion of having multiple user confirmations coupled with a simple way to "cancel and correct" the notice would have likely meant this never became news worthy.

Sure, but that's a lot different from "move these buttons around and have the user double click to be sure."
The extra confirmation shouldn’t have been on the same user. The “two man rule” exists for emergency protocols for a reason.
Confirmation screens might have given the operator a moment's pause to consider whether or not they were certain.

A UI that makes it easy to make a mistake, even when the mistake is intended, and makes it almost impossible to correct that mistake, is a badly designed UI.

> Confirmation screens might have given the operator a moment's pause to consider whether or not they were certain.

There was a confirmation screen.

I have seen no suggestion how a different confirmation than actually existed would have helped, and I don't think we want to introduce more delay in the critical path of a live alert without clear and massive benefit.

> A UI that makes it easy to make a mistake, even when the mistake is intended, and makes it almost impossible to correct that mistake, is a badly designed UI.

The problem with correcting the mistake is not a UI problem, but a deeper planning and procedure problem.

> and I don't think we want to introduce more delay in the critical path of a live alert without clear and massive benefit.

If that delay reduces false alarms, when false alarms erode public trust in and concern about the alert system, then the massive benefit is clear. With such a system, drills and tests are going to be massively more common than actual events. Designing the UI so that the one-in-a-million option is as easy to invoke as the once-every-week option is a UI problem.

Notwithstanding it not being a primary issue in this particular case, I believe it's still an issue worth discussing.

Also, bear in mind that correcting the "deeper planning and procedural problems" would introduce delays as well. Anything but a purely automated system, or a big red button would introduce some delay.

> Also, bear in mind that correcting the "deeper planning and procedural problems" would introduce delays as well

No, it wouldn't, because those are problems that impact after an alert is issued, and whose existence caused delays in sending an all-clear. Resolving them won't effect the path prior to an alert, and would eliminate rather than introduce delays.

How would a UI change have made the outcome different if the employee actually wanted to send a real alert?
When you're doing a post-mortem you're supposed to think about how to prevent the next one, not how to prevent the one that already happened (someone needs to tell the State Dept this).

So you look at all of the contributing factors and you come up with a plan for all the ones you can reasonably address.

It's been identified that a reasonable person could accidentally send a real alert even when they intended to send a fake one. So you fix that too because why the hell would you leave it once you've found it?

Or to put it another way, they're already in the middle of a PR disaster. You get a second one if the press gets wind of the idea that you're cutting stupid simple corners on the remediation. And make no mistake: Fixing a UI issue with the benefit of hindsight is a stupid simple fix (avoiding it to begin with is much harder).

This is a solved problem -- the UI could require confirmation from a second employee.
You have minutes before the missiles hit. Waiting around for Steve to get back from the toilet could cost a lot of lives.
Like I said, the 2-man rule is a solved problem.

https://en.wikipedia.org/wiki/Two-man_rule

What does Bob do when he goes to the toilet? He asks Steve to cover him.

So if you need 2 people to send an alert, when Bob is at the console, and Steve is secondary, when Steve has to go to the toilet, he asks Jill to cover him. Alternatively a backup authorization method could be used.

It does mean having 3 people available at all times, but since there are life-safety implications of real alerts as well as false alerts, it seems like a reasonable safeguard to have.

Actually, it's even more crappy UI, just not in the visual domain. Including the phrase "this is not a drill" in something that actually is a drill could be considered a design flaw in the auditory user interface.
Pedants, all of you.
Hello. Welcome to Hacker News. Enjoy your visit!
(comment deleted)
Was that addressed to me? I certainly wasn't attempting to diss the community at large.

Yes, sometimes the pedantry and "um actually" nature of this place bugs me, but that's just what happens in gatherings of nerds.

OK, I overinterpreted you and will delete my comment. Sorry!

(It was still an unsubstantive post though... please don't post those, even when someone else is setting a bad example.)

(comment deleted)
I know how annoying it can be to run across internet comments that seem insufferably pedantic. But would you please not post unsubstantive comments to Hacker News? As much of a problem as pedantry is, empty, name-calling comments are much worse.

https://news.ycombinator.com/newsguidelines.html

Huh, interesting. Personally I read that as being a kind of wry, collegial, back-handed complement. Wasn't at all taken aback; it put a smile on my face. But perhaps I'm seeing British Humour where there is none. And you're right that it wasn't substantiative, and nobody wants HN to turn into Reddit.

So if earnestness is to rule the day, allow me to say that my point was more serious than mere pedantry. Because most of us here are software people living highly visual environments, and because our thinking is shaped by language that disproportionally based on visual metaphors, we tend to assume that "interfaces" are necessarily visual systems. They are not. They can be auditory and even tactile. If an audio message gives incorrect, contradictory, or badly-structured information, then that's no less an interface failure than a badly-designed web page. In this case, both the audio message and the alert interface shared a common point of failure: a deep, seemingly institutional inattention to how information is presented and interpreted. In both cases, doing a better job with the interfaces is hardly rocket science; that's not the interesting part of this fiasco. The interesting part is: why didn't the institutions care about this, how did the project managers fail to spot the problems, and how should this change procurement and management practices in the future?

Hopefully that's a bit less pedantic!

I like your interpretation better than mine.
Yeah, but I'm privileged. When somebody is definitely being an ass-hat online, I have the luxury of breezily going about my merry and generally not giving a flip. Life is too short for flame wars. So lacking significant exposure to ass-hattery, I tend to optimistically assume that I'm not seeing it. Might be overly innocent, but keeps me happy.

Being a mod has got to be tough: you're not allowed to look away from the assholes; you're required to look directly at them. After dealing with the 10,000th jerkwad, an optimistic reading of what anyone says has got to be challenging. So divergent readings are totally understandable, and thanks for the reminder that when using flippancy or sarcasm online, it's a good idea to telegraph one's amiable intent.

Sure sounds like it!

What is confusing is that the article still points to the UI as being a part of the problem though while they're now saying the employee intentionally sent the live message. Looks like the PR campaign is still trying to deflect blame or perhaps keep things just ambiguous enough that they'll be able to rush through some extra spending to "fix" the UI issue.

Hopefully more details will continue being shared and they deal with the organizational flaws instead of pouring blame on scape goats (software and individuals).

Or, there were a bunch of potential land-mines set in this process, and eventually the employee stepped on one. Which one he stepped on is maybe less interesting to me, than the fact of the organizational culture producing multiple ones to step on.
Yup. I called bullshit on the story we were being fed at the time (https://news.ycombinator.com/item?id=16161584) and said it seemed obvious when you think about it that they really believed they were under attack, and I'm glad to see the truth finally come out. The bad UI argument could still be valid, but it definitely was not the cause of this.
Same here, and I was called a reptilian flat eaether or something to that effect.

I'll never be shamed out of questioning clear nonsense.

Well, we've seen the UI and it is crappy and it's not hard imagine someone making a mistake that way. But yeah you're kinda right. They should still fix the UI though :/.
You can't blame people for making inferences based on the news that was reported.
I mean, I fully suspect that both suck
No. They are getting tripped up in their numerous lies.

Last time this happened, the story simply disappeared from the media altogether. Expect that to happen here too.

> Following standard procedures, the night-shift supervisor posing as Pacific Command played a recorded message to the emergency workers warning them of the fake threat. The message included the phrase “Exercise, exercise, exercise.” But the message inaccurately included the phrase “This is not a drill.”

This is just a test. This is not a test.

Especially since this was in Hawaii. The first message from Pearl Harbor to the fleet during the attack in 1941 was, famously,

“AIRRAID ON PEARL HARBOR X THIS IS NO DRILL.”

“The day shift warning officers received this recorded message on speakerphone. While other warning officers understood that this was a drill, the warning officer at the alert origination terminal claimed to believe that this was a real emergency”

https://www.npr.org/sections/thetwo-way/2018/01/30/581853255...

Speakerphone. That can also mean it’s automatically on. So you do something else, hear something narrated as the message and the last thing that stays in your head is clear “this is not a drill.” And you probably didn’t have to answer the call and talk to the real person. You’ve just heard “this is not a drill.” So you act as you’re trained to do.

It makes me wonder what the message is for a real emergency.

Also, the article mentions the day-shift supervisor thought the test was for the leaving night-shift operators, so he wasn't there to presumably tell his day-shift that there would be a test... but I think that this is not good for them to be warned ahead of time. You want them to be prepared to execute properly either way.

It's the same message the officer heard, just without "Exercise. Exercise. Exercise." at the start. Bad UI.
The worst civil air disaster happened in part because the controller said “you are not cleared for takeoff”. Alas, there was radio interference, so the “not” didn’t get through.

As a consequence, the words “cleared for takeoff” have been removed from the ATC/pilot vocabulary except when actually cleared for takeoff.

Seems it's still a UX problem.

UX is not limited to visual interfaces. UX in audio communication has been improved in several contexts, aviation is one of the most notable, some of the things they use:

- Standard phraseology

- Use key terms only in affirmative phrases (that is, never say "don't takeoff" but rather "keep your position")

- Disambiguate between similar terms (e.g. 'niner' instead of nine as to not confuse with 'five' as well as other reasons https://www.quora.com/In-aviation-why-is-9-pronounced-niner-... )

So who was fired for this? Someone dropped the ball by not communicating the fact that this was a test to the employee who sent the false alarm. That person should be fired.

This reads like a classic tale of government incompetence. If you don't fire people in a situation like this, why should we expect anything to change?

When it comes to ballistic missile warnings, I’d rather err towards false positives over false negatives. Whomever made this mistake is highly unlikely to do so again. Throwing that away seems wasteful, ceteris paribus.
False positives and false negatives are both really bad. Now if there’s an ICBM launched at Hawaii this weekend, no one will believe it.
> False positives and false negatives are both really bad

Everything surrounding nuclear war is bad. Choosing the lesser of two evils is the best we can do, given the domain.

> Now if there’s an ICBM launched at Hawaii this weekend, no one will believe it

No one? My collection from friends in Hawaii is they’re more prepared. Each of them have at least contemplated a plan and a few assembled go bags. In any case, Japan and Korea’s experiences with false alarms don’t support your claim that a few false negatives outweigh a single false negative.

To be clear, I’m not saying false positives are okay. No error is better than error. But if we’re going to have errors, and we are, one kind is better than the other.

> go bags

where are they planning to go?

It's a common mistake to make up "go bags" but forget they're only supposed to last you the 24 hours that it takes to get to a new refuge should your home/wherever else you're in become unsafe. I'm not sure how that works on Hawai'i after a nuclear missile attack, so I'd love to know what the plan is for such an event for these individuals.

Plausible nuclear attack scenarios do not necessarily mean wiping the entire island off the map. Here is a fairly pessimistic scenario:

http://nuclearsecrecy.com/nukemap/?&kt=15000&lat=21.3069444&...

You can also choose "North Korean weapon tested in 2017" from the drop down menu, clear all effects, and click "Detonate". Also note the mountainous nature of Hawaiʻi.

> where are they planning to go?

Shelter, basement...depends on your situation.

I’m in New York and don’t have a go bag. I did look up our nearest shelter, though, and thought about what I’d grab (water, cat, knife, extra glasses, bacitracin, band-aids, battery + cable, nuts, kibble, phone, flashlight, duct tape and masks).

Right, and that's super reasonable! What a lot of people _do_ do however is make elaborate go bags but but then fail to prep their home, which is almost likely the best place to hunker down and stay.
True, but this also means stupid decisions have no consequences and people prone to making stupid decisions stay in places where they can cause harm with these decisions, or even promoted to places where they cause more harm. Now, everybody makes stupid decisions now and then, so single stupid decision is not necessary damning. But a systemic pattern of it should have consequences. Of course, government employment is a bizzaro world, so making it work there may be hard.
Why was this decision stupid?

The nightshift supervisor intended to run a drill at a very inopportune or unexpected time - this is a great time to test! This is the equivalent of testing edges or boundaries.

The report says the person that sent the alarm did not hear the 'Exercise' portion of the message. It seems they were acting entirely rationally given the protocol they were expected to follow.

It surely was a mistake and things can be improved, but I really don't think this was a 'stupid' decision.

> this is a great time to test! This is the equivalent of testing edges or boundaries.

True, if the test were properly prepared. If not, it's the equivalent of setting your own house on fire to test how the fire department works. Not a smart idea.

Also not being able to cancel the alert immediately is the result of specific system design, which also was somebody's decision.

Pressure to fire people isn't the same thing as pressure to fire the _right_ people. I think the former is more common.
Pressure to fix problems is different from either one of those, though it may sometimes include the second.
I hope one of your coworkers takes a screenshot of this comment for the next time you make a mistake based on miscommunication.
If I was the manager in charge of a team that sends test nuclear missile alarms and my faulty process caused a miscommunication, which in turn created a mass panic, I should be fired.

I don't think the employee who did as instructed and sent the alarm out is at fault, nor do I think they should be fired.

I am reminded of the classic story from IBM.

A junior developer makes a mistake that cost the company 10 million dollars. He walks into the office of Tom Watson, the CEO, expecting to get fired. “Fire you?” Mr Watson asked. “I just spent 10 million educating you.”

Perhaps it is possible to enact some change through learning, rather than through firing.

This was not a junior developer who accidentally rm -rf'd some server, this was a process failure that involved multiple people. In the IBM example, had that mistake been the end result of a faulty process designed by a manager, that manager should be fired or at least removed from a position to make process decisions.

Whoever designed the process for sending out the alarms (likely a committee of sorts) should be held accountable for their gross failure. Firing is a efficient way to remove under-performing employees and ensure that remaining employees give the needed attention to the important problems.

Maybe the employee who clicked the button to send the false alarm should be consulted when the alarm process is redesigned. I'm sure that they would be very familiar with what went wrong and how it could be improved.

You think government incompetence is bad, you should see corporate incompetence!
Firing people is not the only answer to errors. It depends on the intent of the person. If the person was neglectful or that it is proven that the person was being irreparably incompetent/lacking judgement, then sure.

Likely, though, this is an error that is more reflexion of a problem in the UI and/or drill process, and investing resources in making sure these are improved would be a better way to prevent further mistakes.

If someone should get fired, it's the person who put "This is not a drill" in the auditory warning that tricked the operator, and the person who didn't put in a "We messed up" text alert button.
Right, my point exactly. The employee who pushed the button was just doing as instructed.
I'm not saying the UI of their system is great, since I have never seen it, but I believe it's a better idea to have the "test" alert be very similar to the real one. When the time comes to send a real alert, the operators should be practiced. Diverging the UI could cause confusion in am actual emergency, wasting precious moments.
(comment deleted)
I think the most critical problem this revealed is not in the chain of events leading to the false alert (which contains several real problems), but that the State of Hawaii apparently has no clear process for sending an all clear and informing the public of the resolution after a live alert.
They probably figured there wasn't going to be a State of Hawaii after a real nuclear missile so why bother with an all clear?
Real missiles can miss, malfunction (the missile), be intercepted, not have a nuclear warhead in the first place, experience a warhead malfunction, and—even if they hit somewhere in the area of Hawaii and detonate a nuclear warhead—leave much of the state intact (Hawaii is, after all, about half as far across, east to west, as the continental US.)

The very last may not produce an all-clear in the short term, but it's likely to at least require lifting the “missile is likely to hit in our area soon” alert in favor of a very different emergency posture, which is a similar need.

Firstly, why would you give an alert if everyone is going to die and there's nothing anyone can do? The reason for the alert is that most people (~98%) will be outside the fatal blast of a N. Korean Nuke assuming that it even hits it's target, which I'm not sure the likelihood of anyway. You need the all clear for the surviving 98% (or 100% of people on a miss or false alarm) who will be continuing to get things together after the attack.

If the US is going to exchange nukes with China or Russia, they could decide to wipe out Hawai'i. But N. Korea has like, what 5-6 nukes? They're not known to be accurate either.

Except for Honolulu Hawai'i is mostly composed of small population centers separated by miles of ocean or valley walls. If N. Korea decided that Hawai'i really had to go, I'd guess they could use their arsenal to kill/injure max: 6 missiles * 20k casualties = 100k casualties total, which would still leave 1.1million people to hear an all clear.

I disagree. The problem of the government lying to the public multiple times, that lack of integrity, is a bigger problem. Because if they cannot be trusted to communicate their operations honestly, then how can we trust them when they tell us later (after being ordered to put in place the "clear process" you're asking for) that they did it?

It's like Warren Buffet's three qualities he looks for in an employee: Intelligence, energy and integrity. https://www.cnbc.com/2017/10/05/character-traits-warren-buff...

> The problem of the government lying to the public multiple times, that lack of integrity, is a bigger problem.

I was talking about problems within the boundaries of the alert incident, and that’s a level removed from the universe within which I was characterizing “the biggest problem”, and it's not clear to me how much was premature communication before all the facts were in and how much was lying, but I don't disagree that the post-incident communications reveal problems of trustworthiness that are potentially a bigger problem than anything within the incident.

Everybody who publicly excoriated the UI (or what they thought was the UI) of the Hawaii public alert system here should think about their role in spreading misinformation. This story was catnip to computer people, and too few people waited for any kind of confirmation before deciding they knew all about what happened, and how to fix it.
It still is bad UI, that hasn't changed. So people didn't spread misinformation.

Sure, the employee clicked the wrong button because of this audio alert, but the buttons/text are still poorly designed.

There's no way to tell if it's bad UI given the publicly available information. We'd need to see the whole flow (including confirmation steps), as well as know something about how the employees were trained on this interface.
That's UX, not UI. The user interface is bad. The bad labelling/naming of the links makes it more likely to click the wrong button. That is bad UI. Bad UX would be the situation if even after that, there is no double check or alert to confirm that users intention.
I wouldn't call spreading misinformation because that was the official statement. It just turned out that the official statement was incorrect.

Besides, memes were brilliant :)

After a few more days, we will hear that it was not the worker who sent the alert, and after some more days, we will hear that nobody sent the alert and that it was a bug. Just kidding. News such as these have to wait till the investigation is complete, otherwise we will simply be reading speculation.
Why does the state of Hawaii even need a man in the middle for issuing ICBM alerts. It would seem the military or at least something centralized at the federal level should be able to trigger this alert. Basically have it patched through directly.

I can't see a benefit of an extra person in the loop. It could at best delay sending the alert, send the wrong alert. At worst is obviously missing sending the alert altogether. One might say, well one benefit is the state could prevent a bogus alert from being sent. But does the State of Hawaii have its own independent monitoring network of satellites or radar? I am guessing it doesn't.

I can see value in having someone capable of issuing alerts who is geographically _on_ Hawaii, as the potential for dropped communication lines seems problematic.

I don't know enough about military/federal operations to know whether having them handle it would imply keeping operations on the islands, or if they'd literally "centralize" it to somewhere on the continental US.

So, more fundamental than UX, that the proper employee action for an “exercise” differs from that for a real threat doesn’t seem to be a proper test. Why not just have an administrative airgap keyswitch that enables exercises to be conducted without the requirement to disseminate the fact beforehand? Then you can test for the proper response without risk.