> (The computer symbol for swearing, #$@&%*!, was chosen because it’s the output when you’re trying to type numbers but you’ve inadvertently engaged the caps lock key.)
Did this used to be true? I have never encountered this. Holding down Shift, perhaps, but not caps lock...
I've never used a keyboard/driver which had this functionality (CAPS Lock only affects letters on any system I've used in the past). However, I believe the SHIFT LOCK key that existed on some typewriters functioned in the fashion described in the article, as the SHIFT LOCK key did, in fact, affect all keys on a few typewriters I have used.
Serious cliff hanger on this article, with the second part of how it actually made the mistake held off until tomorrow. Reminds me of one of those old tv shows...
UX design, especially for medical software, is incredibly important and yet is rarely done by people who actually understand how to do it.
I remember when my mom had radiation treatments for brain cancer, which were so awful they killed her quicker than the disease would have (by a few weeks). A few months later I read about how terrible the design for calibrating these machines was leading to a large number of mis-treatments, my sister and I wondered what dose she actually received.
During the Vietnam war pilots turned off all the alarms in some of their planes due to constant alarms that always happened during intense combat situations leading to an inability of focus on what mattered.
UX design is not easy but has to be treated as more important than just making it pretty. Understanding how real people use these things is incredibly important and not just a last minute bullet check.
>>>UX design is not easy but has to be treated as more important than just making it pretty. Understanding how real people use these things is incredibly important and not just a last minute bullet check.
Amen brother.
The last four large scale business enterprise applications I've worked on, design was left to either the back-end developers, project managers or done by committee. The ones done by committee were absolutely brutal. Nothing rips your soul out quicker than trying to get 20 or more people to agree on something.
I have no idea when or where design just got completely buried and left to people who clearly have no idea what they're doing, but in your case, it's incredibly troubling and scary.
Yes, my grandmother received approximately 100x the dose she was supposed to receive due to operator error which resulted in her intestines being fried and gave her digestive problems for the rest of her life. (Did kill the cancer though)
Unfortunately medical software is enterprise software, because Hospitals are large, risk adverse institutions with high price tags and purchasing is done by management not users. Like most enterprise class software UX is a complete after thought.
A redeeming factor, at least in software for Medical Devices, is that the FDA recognizes that Use Error is a "thing" and that manufacturers must be aware of ways that an operator may be confused and misuse the product and provide mitigations for them (RTFM is considered an absolute last resort).
We're not allowed to just blame it on the luser and get away with it any more, and that's a Good Thing.
> During the Vietnam war pilots turned off all the alarms in some of their planes
There was Duke Cunningham[1], a fighter pilot ace that (IIRC from a documentary) also turned off the comm link from his RIO to avoid the potential distraction. He was able to speak to his RIO, he just wasn't interested in what the RIO had to say to him. Wow.
His political career later went down in spectacular flames.
> Computer experts call this type of problem — when the same action can result in two very different results — a “mode error,” and it is especially problematic when the user is not focused on the mode (in this case, mg versus mg/kg)
Terrifying example of one way AI or computer-augmented decisions can go horribly wrong. A "Mode error" is at the core of the difference between intelligence and common sense.
FTA: "There is no difference between a minuscule overdose — going 0.1 milligram over a recommended dose — and this very large overdose. They all look exactly the same."
That here is the problem, right here. If we can have self-driving cars (which can look ahead and figure out the result of their actions), so can a system like EPIC. Why can't it calculate the probability of a Bad Thing(tm) happening, and suitable tag the alerts?
Maybe this is just a case of comparing a tech demo (the car) against a deployed system. Presumably hospital software will gradually improve in the future.
While I absolutely agree computers, UIs, and systems in general can stand between a patient and an error, can we just stop and acknowledge how incompetent the doctor was for prescribing 38 instead of 1 double strong dose?
The UI screenshot in the article makes that mistake VERY clear, and their "are you 100% sure" question is obviously intended to illicit a particular response (who is ever really 100% sure of anything?).
I know this article is about how systems can be altered to protect patients, but much like with the NTSB's findings, sometimes it legitimately is pilot error (or doctor error in this case).
Can things be better? Yes. Absolutely. But so can doctors. These types of systems are stop-gaps, without them this doctor could have calculated the wrong dose by hand and made just of large of a mistake.
> and their "are you 100% sure" question is obviously intended to illicit a particular response (who is ever really 100% sure of anything?).
When you routinely get prompted with that question dozens (hundreds?) of times a day, it can't elicit that response successfully. It's just not how our psychology works.
You should actually post a specific criticism. I did read the article, that is my take-away. I'd respond to your issues with my post but you forgot to let us know what they are...
Yes it is obviously an error by the doctor and the pharmacists, no one is saying otherwise. The whole point of the article is to investigate how this error occurred. We live in a world where these errors happens all the time and the best thing we can do is learn from them and if the error analysis boils down to "incompetent doctor", then it is likely that the same error will occur again.
Yes, the doctor and pharmacist and administering nurse failed here. But that is not useful. The useful bit is why they failed, and that seems to be a mix of "busy" with "software sucks".
The doctor didn't order 38.5 tablets -- at least she didn't think she was. She thought she was ordering a dose per kg, not a quantity of tablets.
You say the screen is very clear. It's clear enough when I read it. I had a good night's sleep and I am alert and awake. I've been primed by the article to look for the error. I imagine I would easily miss the mistake if I've been working hospital wards for the last 10 hours on a rolling nightshift, looking after some very ill patients.
The fact that it's possible to use two entirely different dosing systems to order medication and have the screen not clearly indicate[1] which is being used is so obviously sub-optimal I'm kind of surprised to see anyone saying that it's "just pilot error".
> She thought she was ordering a dose per kg, not a quantity of tablets.
Actually, she thought she was ordering a single dose at 160mg, and accidentally ordered dose per kg. 160 was the size of the dose, so the patient's weight became the number of pills.
If someone works with a piece of software every day, they're going to start navigating it by the position of elements in the software, not what the text around the elements say. If the "erase form" button was placed below and to the right of a web form, where the "submit" button usually is, you bet it would cause a fair amount of swearing.
If the doctor is used to working with older/larger patients (over 40kg) they might get that screen only rarely, and perhaps because the patient was so close to 40kg, she wasn't thinking in "pediatrics" mode, and expecting it to come up.
Give me a break. The system has alerts and it spells out the exact dosage in tablets in a dozen places, which is then apparently seen by at least three people.
If these people in the course of their work choose to ignore alerts, are distracted and not actually doing what they were hired to do (hint: theres a reason there were three humans in the chain here), the fault lies on them, not on the software.
You obviously can't have it both ways. We call it automation because things will happen automatically. At the end of the day, this system probably causes significantly less problems than the old way of trying to read some doctors handwriting only to pick up the wrong medicine regardless.
Don't forget about the unwritten 'real policy', as opposed to the actual policy:
>Every training program has a “hidden curriculum” (the way things are actually done around here, as opposed to what the policies say or what the administrators told you during that interminable orientation). One of them — passed down from senior residents to the newbies — was, “Ignore all the alerts.”
Really, you think the UX of those screens is in anyway clear or that the alerts shouldn't be color coded or differentiated by thresholds from "hey this looks like it might be an error" to "this is out of the realm of possibility, do not pass go, do not collect 200, correct this immediately"
But who gets to set the threshold? One doctor might want a full stop at 3% over, one might want one at 10%, one might not necessarily want one until maybe 50% (perhaps in this case of a research hospital that is routinely trying experimental treatments).
I don't disagree that this shouldn't be considered carefully, but it's a little harder than maybe you're making it seem.
Oh, I think it's incredibly complex. I think UI/UX is a huge difficult field and these things always have tradeoffs - not only in terms of usability but in the overhead of being the people tasked with building them. In fact, UI/UX may be the hardest set of tasks we now face as as people making software products that other people use.
But if you think /any/ information is easy to see in that rat's next of anti-design, I think you're wrong. I
All I'm seeing is that the salient information is on the UI in at least three places. It spells out the freaking calculation.
So clearly, these people simply didn't look. Not once. In their perpetual multitasking frenzy that any hospital staff seems to be in, they didn't do the actual task. I'm reasonably sure that you can color code or flash whatever yo u want and they would still miss it if that's the way they work.
A graphical representation of the dose relative to the patient weight or the dose prescribed relative to the recommended dose would be a great way to help users from accidentally making this mode error.
Not true. Alerts are both distracting, causing fragementation of focus and attention, and eventually mentally filtered out. Our brains are wired to ignore stimiuli that are not relevant to our current focus. It's very unwise to expect people to be able to consciously override nature all the time.
I still don't understand how the nurse administering the dose would not have been alarmed that the kid was apparently supposed to swallow almost 40 pills at once. That's the sort of unusual situation you WOULD expect a human to notice right away.
I get it, you start ignoring alerts when theres so many of them. That's also the moment you go back to whatever manufacturer and tell them that you can't work with a safety critical software that will emit spurius alerts to no end.
What you can not do responsibly is to simply ignore all of them and then claim that somehow it's the fault of the software. Yes, as it turns out, you actually take on responsibility when you work in these kind of areas. I personally would never agree to code for systems that end up radiating humans or other safety critical systems.
> I get it, you start ignoring alerts when theres so many of them. That's also the moment you go back to whatever manufacturer and tell them that you can't work with a safety critical software that will emit spurius alerts to no end.
Who do you think buys the equipment or has contact with the vendor?
> If these people in the course of their work choose to ignore alerts
This is where your mistake is coming from. You think for some reason that people are choosing to ignore the alerts, rather than being habituated to ignore them.
It is theoretically possible for a person with a decade or so of training to maintain a constant level of attention on repetitive trivia. Studies on Buddhist monks have shown this. But the people who were doing this didn't study Buddhist meditation, they studied medicine, so expecting them to behave like Buddhist monks, which you seem to be doing, is the height of irrationality.
Humans have very limited attentional resources and extremely powerful internal non-conscious filters to conserve those resources that can only be overcome by lengthy and rigorous training, which would exclude the kind of training these people actually needed to do the rest of their job. These are just facts, well-established by a wealth of empirical research over the past several decades.
Responding "Give me a break" to a system whose failure was fully predictable based on known facts of human psychology and physiology is precisely the problem. Perhaps someone on the design team pointed out the issues, and were told by someone who knew nothing about human psychology beyond what their gut told them, "Give me a break!"
> Like many other physicians, pharmacists, and nurses, Jenny Lucca found alerts to be a constant nuisance. Even giving Tylenol to a feverish child every four hours triggered an alert that the dose was approaching the maximum allowed.
You know, given what we know about the liver toxicity of acetaminophen/paracetamol at this point, not sure this is the best example of a spurious alert. There's a good argument that this is one of the more dangerous drugs OTC, in part due to the widespread impression that it's not very dangerous.
It is interesting looking at the story in general as an example of alert fatigue, though, and the need to differentiate different types of alerts.
Unlike some of the other comments I see here, I don't think this points to incompetence so much as human nature. It takes a fair amount of cognitive power to overcome the natural tendency to assume that when the alert was spurious yesterday, it's not spurious today.
That power would be better spent elsewhere, and points to a failure of automation to truly lighten the load.
Not "exceeding the maximum dose" just "approaching the maximum dose".
"Hey, you're almost taking exactly the amount of this thing we say you can take. Watch out!" is pretty much the definition of a spurious alert to me. It's telling you that what you expect to happen is happening.
Well, yeah, except pills up the amount in steps--it's not like turning a dial. So you'd want to warn before they can give another dose of medication that would take you over the line. Really depends on how far off the line the warning is.
But either way, in this case I'm pretty sure Tylenol was stressed in the article because it was an example of an innocuous medication, which it really kind of isn't.
I work developing an internal company app that's already a little trigger-happy on the e-mail alerts (which are not at all life-and-death) and a lot of it boils down to an organizational/political malaise.
The most common problem is when the people setting alert policies are divorced from the people receiving them. The managers have no direct incentive to keep the alerts sane or moderated. On an individual basis, every alert that exists shifts potential blame from the manager to the employee. ("It's all the employee's fault for not following the alerts.")
In this way, over-alerting is often "automated micromanagement", but without even the feedback-loop and mutual-accountability of actual micromanagement. Until the people acting on alerts have some feedback into the alert-configuring process, the problem will continue.
Another dysfunction is when people ask for alerts purely for an independent paper trail, either because they're paranoid about assigning blame, or because they want to use their e-mail client as a half-assed searchable database.
P.S.: We've already had managers asking us if we could have e-mail alerts that users must confirm receipt of by visiting the website... and for the site to continuously e-mail followups until they do. Meanwhile, the IT staff have their own horror-stories of ancient inboxes grown too large to reliably read data from.
This is exactly it--nurses are usually not empowered to set alarm thresholds or delays, but physicians are. So, docs ask for the most sensitive settings they can (because they want that intimacy of care), but they're gone 15 minutes later and the rest of the unit has to deal with the fallout for the rest of the day.
Not sure what "intimacy of care" means, but most docs, like me, are frustrated and annoyed by the barrage of wolf-crying that comes out of these systems, of which Epic is one of the best and best-designed. (So far, all the others I've seen are FAR worse.) I WANT to see the meaningful alerts, but it's difficult to do with a sea of identical-appearing utter BS. Besides alert fatigue, there's a related problem with cognitive load. For this patient, I want to order A, B, C, and D. Now erect roadblocks in front of A, B, and C, and quite often D will fall by the wayside. By the way, I've never been able to set or even influence the alert policy. This was mandated by corporate risk management, whose policy appears always to be more alerts = safer.
We've seen alarm policies range from "Dr. X wants this to be the threshold on the patient" to "Unit policy is X, change during a PDSA cycle someday" to "Eh, this seems alright for this patient" to "silence as many alarms as reasonable".
I'm sympathetic to "it's corporate's fault", but instead of fixing that problem I see a lot of folks lobbying for shiny new toys, meetings to stroke egos, and basically spending their political capital on being prima donnas instead of making their lives easier from an operations standpoint--and for the rest of the docs such as yourself this obviously is a problem. :(
The roadblocks bit is what it is--if doctors were better at prescribing medications and hadn't cowed their colleagues and nurses into a culture of "oh, well, I shouldn't say anything" then automated systems wouldn't be so compelling from a safety standpoint. As it is, they are undermined continually (as explained in this article) by the humans who have to work with them and who--as you've exemplified--don't trust them.
Let's face it: in the next 30 years, most physicians who are basically knowledge workers are going to be doing work very differently than they are today, most of it with heavy computerized assistance. You can choose which side of history you want to be on.
First, Epic does a lot of custom work--as noted in the article, each installation is "customized" to only have particular alerts. This makes it hard to count on a standard system behavior, because there isn't one. This continued coddling of clients makes it very hard to introduce new software into the market, because you can't do it without deep pockets and sloooooow sales cycles.
Second, the physicians are very much overloaded with alarms and alerts--and instead of saying "Hey, let's figure out why we keep ending up with so many alarms" they instead do the short-term thing and just ignore or turn them off. The entire profession is rooted in split-second decisionmaking, for reasons I cannot fathom. They really, really, really want to be the heros that come in and save the day, and systematic approaches to operations optimization just aren't a priority.
Third, the data coming off of these things (and at work we've seen this) usually indicates large operational deficiencies. Unfortunately, you can't just tell the clients "Hey, you all are doing things wrong, and here's the proof", because they either won't care, or they will fire you and bring in somebody who keeps them happy, or they'll care but be unable to make changes because that introduces uncertainty into their line of work and that could hurt someone--nevermind all the existing casualties.
It's more fundamental than that, I think. It's core to the entire mindset of the profession.
If you compare the operators of decades ago with the sysadmins of the 90s and the devops folks of today you can see a rough path of evolution that hopefully medicine will follow: a last-minute heroic effort to save a system is a failure, not a victory.
"The entire profession is rooted in split-second decisionmaking, for reasons I cannot fathom."
Right. There are very few points in time where decisions in medicine need be like this. Even in the ED, where I interact most with physicians. Hell, the ED I see most often has checklists to "stop and review" (and they do) for things that most people would consider (and are) time-critical, cardiac arrests, intubation, and the like. It's odd to get used to, but seems to work very well.
Confused about one thing, did the robot put 38.5 pills in a bottle, or did it formulate a pill containing 38.5 pills worth of meds into one pill, which the resident gave to the patient? I'm thinking sitting there and giving the patient 38.5 pills for the patient to swallow would have been considered unusual?
From my experience with pharmacy robots (my wife owns a Pharmacy, their group have just decommissioned their expensive robot because it doesn't work):
It almost certainly put 38.5 pills per day in a bottle. At which point the dispensing Pharmacist or Assistant should have started asking questions.
Doctors -up these kind of things all the time; around 5% of prescriptions based on the very scientific measurement of the relative size of the good/bad stacks on my wife's desk.
It attached them in 39 little bags to a loop of plastic. You can see the picture near the bottom. If I see 39 individual pills in 39 individual bags like that and am being told that I am to administer them, at once, to a patient, alarm bells in my head are ringing, as a paramedic. Swallowing 39 tablets like that is not fun for anyone, even aside from risks of OD.
I'm wondering about this too. I guess we'll have to wait until Part 3.
One other thing that went wrong here - through no fault of her own, the mom was in a different room with the brother. She knows what the patient normally takes, and would likely have started asking questions when she saw her son taking way more than what he usually took.
I suspect a great deal of lurking complexity and privacy concerns there. What's spurious in one case is critical in another, and the difference lies in HIPAA-sensitive data.
I'm torn on this. On one hand: we can see the process, and clearly identify a number of mistakes in the software that led to this problem. Serious mistakes, that need to get fixed. And on the other: while this was an error that wouldn't happen without computers and robots, it's clear that this process will reduce the number of overall errors, saving peoples lives. But it's a different set of people.
This is similar to the "Trolley Problem" ethical thought experiment—we have a computerized system, that will save a lot of peoples' lives—at the cost of others, who wouldn't have otherwise died. It's a hard problem, with no easy solution. See: http://en.wikipedia.org/wiki/Trolley_problem
There are least two comments here already ascribing the error to incompetence; I expect more will come.
In my opinion, this is a dangerous misperception. Hindsight is 20/20. You saw one screenshot displaying the incorrect dosage; the doctors see "scores" each day, day in, day out. The brain is hardwired to ignore repetitive stimuli. Humans are smarter than computers overall but have a much higher error rate; "competence" can decrease that rate, but not to zero. There is always a chance of a mistake.
The software could have been designed to make the probability of requesting a 38.5x overdose from the robot pharmacist negligible, by reserving a special warning for extreme circumstances. This requires the programmers to do a non-shit (shall I say "competent"?) job once, but not constant vigilance by the doctor, who I'm sure can use the saved mental cycles to check for many other potential errors the machine can't handle completely trivially.
I expect a more thorough design overhaul would cause a non-negligible reduction in the occurrence of a wide variety of mistakes, dangerous and not.
There was a remark made by one of those involved, in the article, that EPICs alerting didn't differentiate between a 'trivial' alert, say "Dose is 1.02x maximum recommended dosage" and a much more critical "Dose is 38.5x..."
Also for mg vs mg/kg, that's a harder problem - off the top my head, get rid of the modes, and show both mg/kg and mg textboxes, with a radio button to select the one you want to edit. Edit it, and the non-selected text box is grayed-out and uneditable, and shows you the corresponding calculation (enter in mg/kg, and it shows you the mg amount of medicine; enter in mg, and it shows you the mg/kg ratio).
> The software could have been designed to make the probability of requesting a 38.5x overdose from the robot pharmacist negligible, by reserving a special warning for extreme circumstances
A special warning is good. But the software could ALSO require the doctor to, instead of just clicking, read a message like this on the screen:
YES I REALLY WANT A DOSE OF 6160 MG OF (whatever)
Then force the doctor to re-type that exact message into an input field. Typing required, no cut/paste allowed.
There is nobody, no matter how busy, who would type those words without thinking about what they are doing.
Of course, that sort of typing can't be requested routinely, only in extreme circumstances. Perhaps these extreme circumstances would require TWO people to type that same message?
Or, maybe the software should simply REJECT a dosage greater than perhaps 10x safe dosage for a person. No override allowed?
Pharmacists shouldn't be "clicking through" alerts. They should have a separate, joystick style device with an actual trigger (bright red). That trigger, and that trigger alone, can dismiss an alert.
Seriously, what would that cost, like $20? (Obviously there are a lot more things wrong with this system than just using a mouse for too much, but this would help). We should be willing to invest more in our machines: http://www.loper-os.org/?p=861
And if they are beset with hundreds of mostly innocuous alerts a day, the low signal/noise ratio of the alerts and their sheer quantity will train the pharmacists to press that button just a bit more blithely each time. That leads one the believe the correct course of action is to reduce the number of alerts but then that was at least part of the cause of the incident related in the article.
My wife is a Pharm.D. at a hospital (that coincidentally is currently installing Epic). I know she worries (obsesses even) over the possibility of making an error that hurts or kills someone. I'd like to think other people doing that work feel the same way. Even if she can get three or four nines worth of confidence in every decision she makes however, that leads to some unfortunate probabilities when multiplied out by a career's worth of decisions. We can improve the technology to reduce the chances of significantly negative outcomes but they will always be non-zero.
I don't envy her or the programmers working on systems like Epic. It's messy work with serious consequences.
This is why you must always always be at your kids beside and if you are in the hospital, always have a friend with you (if you are in the hospital and otherwise not capable), who can double check and ask questions on your behalf every time.
My proposal: usability experts should identify an acceptable range of alerts on a per-order, per-encounter basis. End-users of all disciplines should have the opportunity to provide feedback in real-time, in the live environment. Just like Amazon reviews, "Was this alert helpful?" If you always vote "no", your feedback will be down-weighted. With a few exceptions, like wrong patient or 10x overdose, all the other alerts will be ranked in order of utility and only the top x% will be shown.
70 comments
[ 4.2 ms ] story [ 135 ms ] threadDid this used to be true? I have never encountered this. Holding down Shift, perhaps, but not caps lock...
http://ux.stackexchange.com/questions/64230/why-does-the-cap...
Not sure about the Caps Lock thing though.
I remember when my mom had radiation treatments for brain cancer, which were so awful they killed her quicker than the disease would have (by a few weeks). A few months later I read about how terrible the design for calibrating these machines was leading to a large number of mis-treatments, my sister and I wondered what dose she actually received.
During the Vietnam war pilots turned off all the alarms in some of their planes due to constant alarms that always happened during intense combat situations leading to an inability of focus on what mattered.
UX design is not easy but has to be treated as more important than just making it pretty. Understanding how real people use these things is incredibly important and not just a last minute bullet check.
Amen brother.
The last four large scale business enterprise applications I've worked on, design was left to either the back-end developers, project managers or done by committee. The ones done by committee were absolutely brutal. Nothing rips your soul out quicker than trying to get 20 or more people to agree on something.
I have no idea when or where design just got completely buried and left to people who clearly have no idea what they're doing, but in your case, it's incredibly troubling and scary.
We're not allowed to just blame it on the luser and get away with it any more, and that's a Good Thing.
Most programmers I meet don't even understand the importance of it looking nice. They think that the user should do their due diligence. :(
There was Duke Cunningham[1], a fighter pilot ace that (IIRC from a documentary) also turned off the comm link from his RIO to avoid the potential distraction. He was able to speak to his RIO, he just wasn't interested in what the RIO had to say to him. Wow.
His political career later went down in spectacular flames.
[1] https://en.wikipedia.org/wiki/Duke_Cunningham
Terrifying example of one way AI or computer-augmented decisions can go horribly wrong. A "Mode error" is at the core of the difference between intelligence and common sense.
That here is the problem, right here. If we can have self-driving cars (which can look ahead and figure out the result of their actions), so can a system like EPIC. Why can't it calculate the probability of a Bad Thing(tm) happening, and suitable tag the alerts?
The UI screenshot in the article makes that mistake VERY clear, and their "are you 100% sure" question is obviously intended to illicit a particular response (who is ever really 100% sure of anything?).
I know this article is about how systems can be altered to protect patients, but much like with the NTSB's findings, sometimes it legitimately is pilot error (or doctor error in this case).
Can things be better? Yes. Absolutely. But so can doctors. These types of systems are stop-gaps, without them this doctor could have calculated the wrong dose by hand and made just of large of a mistake.
When you routinely get prompted with that question dozens (hundreds?) of times a day, it can't elicit that response successfully. It's just not how our psychology works.
The doctor didn't order 38.5 tablets -- at least she didn't think she was. She thought she was ordering a dose per kg, not a quantity of tablets.
You say the screen is very clear. It's clear enough when I read it. I had a good night's sleep and I am alert and awake. I've been primed by the article to look for the error. I imagine I would easily miss the mistake if I've been working hospital wards for the last 10 hours on a rolling nightshift, looking after some very ill patients.
The fact that it's possible to use two entirely different dosing systems to order medication and have the screen not clearly indicate[1] which is being used is so obviously sub-optimal I'm kind of surprised to see anyone saying that it's "just pilot error".
Actually, she thought she was ordering a single dose at 160mg, and accidentally ordered dose per kg. 160 was the size of the dose, so the patient's weight became the number of pills.
If the doctor is used to working with older/larger patients (over 40kg) they might get that screen only rarely, and perhaps because the patient was so close to 40kg, she wasn't thinking in "pediatrics" mode, and expecting it to come up.
If these people in the course of their work choose to ignore alerts, are distracted and not actually doing what they were hired to do (hint: theres a reason there were three humans in the chain here), the fault lies on them, not on the software.
You obviously can't have it both ways. We call it automation because things will happen automatically. At the end of the day, this system probably causes significantly less problems than the old way of trying to read some doctors handwriting only to pick up the wrong medicine regardless.
>Every training program has a “hidden curriculum” (the way things are actually done around here, as opposed to what the policies say or what the administrators told you during that interminable orientation). One of them — passed down from senior residents to the newbies — was, “Ignore all the alerts.”
I don't disagree that this shouldn't be considered carefully, but it's a little harder than maybe you're making it seem.
But if you think /any/ information is easy to see in that rat's next of anti-design, I think you're wrong. I
So clearly, these people simply didn't look. Not once. In their perpetual multitasking frenzy that any hospital staff seems to be in, they didn't do the actual task. I'm reasonably sure that you can color code or flash whatever yo u want and they would still miss it if that's the way they work.
I still don't understand how the nurse administering the dose would not have been alarmed that the kid was apparently supposed to swallow almost 40 pills at once. That's the sort of unusual situation you WOULD expect a human to notice right away.
What you can not do responsibly is to simply ignore all of them and then claim that somehow it's the fault of the software. Yes, as it turns out, you actually take on responsibility when you work in these kind of areas. I personally would never agree to code for systems that end up radiating humans or other safety critical systems.
Who do you think buys the equipment or has contact with the vendor?
This is where your mistake is coming from. You think for some reason that people are choosing to ignore the alerts, rather than being habituated to ignore them.
It is theoretically possible for a person with a decade or so of training to maintain a constant level of attention on repetitive trivia. Studies on Buddhist monks have shown this. But the people who were doing this didn't study Buddhist meditation, they studied medicine, so expecting them to behave like Buddhist monks, which you seem to be doing, is the height of irrationality.
Humans have very limited attentional resources and extremely powerful internal non-conscious filters to conserve those resources that can only be overcome by lengthy and rigorous training, which would exclude the kind of training these people actually needed to do the rest of their job. These are just facts, well-established by a wealth of empirical research over the past several decades.
Responding "Give me a break" to a system whose failure was fully predictable based on known facts of human psychology and physiology is precisely the problem. Perhaps someone on the design team pointed out the issues, and were told by someone who knew nothing about human psychology beyond what their gut told them, "Give me a break!"
You know, given what we know about the liver toxicity of acetaminophen/paracetamol at this point, not sure this is the best example of a spurious alert. There's a good argument that this is one of the more dangerous drugs OTC, in part due to the widespread impression that it's not very dangerous.
It is interesting looking at the story in general as an example of alert fatigue, though, and the need to differentiate different types of alerts.
Unlike some of the other comments I see here, I don't think this points to incompetence so much as human nature. It takes a fair amount of cognitive power to overcome the natural tendency to assume that when the alert was spurious yesterday, it's not spurious today.
That power would be better spent elsewhere, and points to a failure of automation to truly lighten the load.
"Hey, you're almost taking exactly the amount of this thing we say you can take. Watch out!" is pretty much the definition of a spurious alert to me. It's telling you that what you expect to happen is happening.
But either way, in this case I'm pretty sure Tylenol was stressed in the article because it was an example of an innocuous medication, which it really kind of isn't.
Why is acetaminophen even in the picture? Isn't ibuprofen very similar in effect but much safer in an overdose?
Can't speak for why it'd be favored in children, but NSAIDs do conflict with a number of medicines used in adults.
The most common problem is when the people setting alert policies are divorced from the people receiving them. The managers have no direct incentive to keep the alerts sane or moderated. On an individual basis, every alert that exists shifts potential blame from the manager to the employee. ("It's all the employee's fault for not following the alerts.")
In this way, over-alerting is often "automated micromanagement", but without even the feedback-loop and mutual-accountability of actual micromanagement. Until the people acting on alerts have some feedback into the alert-configuring process, the problem will continue.
Another dysfunction is when people ask for alerts purely for an independent paper trail, either because they're paranoid about assigning blame, or because they want to use their e-mail client as a half-assed searchable database.
P.S.: We've already had managers asking us if we could have e-mail alerts that users must confirm receipt of by visiting the website... and for the site to continuously e-mail followups until they do. Meanwhile, the IT staff have their own horror-stories of ancient inboxes grown too large to reliably read data from.
I'm sympathetic to "it's corporate's fault", but instead of fixing that problem I see a lot of folks lobbying for shiny new toys, meetings to stroke egos, and basically spending their political capital on being prima donnas instead of making their lives easier from an operations standpoint--and for the rest of the docs such as yourself this obviously is a problem. :(
The roadblocks bit is what it is--if doctors were better at prescribing medications and hadn't cowed their colleagues and nurses into a culture of "oh, well, I shouldn't say anything" then automated systems wouldn't be so compelling from a safety standpoint. As it is, they are undermined continually (as explained in this article) by the humans who have to work with them and who--as you've exemplified--don't trust them.
Let's face it: in the next 30 years, most physicians who are basically knowledge workers are going to be doing work very differently than they are today, most of it with heavy computerized assistance. You can choose which side of history you want to be on.
First, Epic does a lot of custom work--as noted in the article, each installation is "customized" to only have particular alerts. This makes it hard to count on a standard system behavior, because there isn't one. This continued coddling of clients makes it very hard to introduce new software into the market, because you can't do it without deep pockets and sloooooow sales cycles.
Second, the physicians are very much overloaded with alarms and alerts--and instead of saying "Hey, let's figure out why we keep ending up with so many alarms" they instead do the short-term thing and just ignore or turn them off. The entire profession is rooted in split-second decisionmaking, for reasons I cannot fathom. They really, really, really want to be the heros that come in and save the day, and systematic approaches to operations optimization just aren't a priority.
Third, the data coming off of these things (and at work we've seen this) usually indicates large operational deficiencies. Unfortunately, you can't just tell the clients "Hey, you all are doing things wrong, and here's the proof", because they either won't care, or they will fire you and bring in somebody who keeps them happy, or they'll care but be unable to make changes because that introduces uncertainty into their line of work and that could hurt someone--nevermind all the existing casualties.
Because insurance companies, Medicare, and Medicaid reimbursements barely cover costs, so to be profitable they have to make it up on volume.
If you compare the operators of decades ago with the sysadmins of the 90s and the devops folks of today you can see a rough path of evolution that hopefully medicine will follow: a last-minute heroic effort to save a system is a failure, not a victory.
Right. There are very few points in time where decisions in medicine need be like this. Even in the ED, where I interact most with physicians. Hell, the ED I see most often has checklists to "stop and review" (and they do) for things that most people would consider (and are) time-critical, cardiac arrests, intubation, and the like. It's odd to get used to, but seems to work very well.
It almost certainly put 38.5 pills per day in a bottle. At which point the dispensing Pharmacist or Assistant should have started asking questions.
Doctors -up these kind of things all the time; around 5% of prescriptions based on the very scientific measurement of the relative size of the good/bad stacks on my wife's desk.
One other thing that went wrong here - through no fault of her own, the mom was in a different room with the brother. She knows what the patient normally takes, and would likely have started asking questions when she saw her son taking way more than what he usually took.
If we can filter out the Nigerian prince scams from your email, why not apply that to save lives?
This is similar to the "Trolley Problem" ethical thought experiment—we have a computerized system, that will save a lot of peoples' lives—at the cost of others, who wouldn't have otherwise died. It's a hard problem, with no easy solution. See: http://en.wikipedia.org/wiki/Trolley_problem
In my opinion, this is a dangerous misperception. Hindsight is 20/20. You saw one screenshot displaying the incorrect dosage; the doctors see "scores" each day, day in, day out. The brain is hardwired to ignore repetitive stimuli. Humans are smarter than computers overall but have a much higher error rate; "competence" can decrease that rate, but not to zero. There is always a chance of a mistake.
The software could have been designed to make the probability of requesting a 38.5x overdose from the robot pharmacist negligible, by reserving a special warning for extreme circumstances. This requires the programmers to do a non-shit (shall I say "competent"?) job once, but not constant vigilance by the doctor, who I'm sure can use the saved mental cycles to check for many other potential errors the machine can't handle completely trivially.
I expect a more thorough design overhaul would cause a non-negligible reduction in the occurrence of a wide variety of mistakes, dangerous and not.
A special warning is good. But the software could ALSO require the doctor to, instead of just clicking, read a message like this on the screen:
Then force the doctor to re-type that exact message into an input field. Typing required, no cut/paste allowed.There is nobody, no matter how busy, who would type those words without thinking about what they are doing.
Of course, that sort of typing can't be requested routinely, only in extreme circumstances. Perhaps these extreme circumstances would require TWO people to type that same message?
Or, maybe the software should simply REJECT a dosage greater than perhaps 10x safe dosage for a person. No override allowed?
Seriously, what would that cost, like $20? (Obviously there are a lot more things wrong with this system than just using a mouse for too much, but this would help). We should be willing to invest more in our machines: http://www.loper-os.org/?p=861
My wife is a Pharm.D. at a hospital (that coincidentally is currently installing Epic). I know she worries (obsesses even) over the possibility of making an error that hurts or kills someone. I'd like to think other people doing that work feel the same way. Even if she can get three or four nines worth of confidence in every decision she makes however, that leads to some unfortunate probabilities when multiplied out by a career's worth of decisions. We can improve the technology to reduce the chances of significantly negative outcomes but they will always be non-zero.
I don't envy her or the programmers working on systems like Epic. It's messy work with serious consequences.