More than half the erroneous letters were sent to customers who hadn't had their blood drawn yet for the Galleri test, the spokeswoman added.
That would probably invoke an "I'm being phished" reaction if I were in that situation, but I do wonder if the average person would realise the impossibility of the situation.
More and more I think "software" and "developer" are over-broad terms. "Move fast and break things" can be fine for projects where errors don't cause significant harm. But it seems dangerous to me to think that job's the same as one where it becomes "move fast and break people".
Did you? I have seen people get told they have cancer. That Grail can blame some other company is surely little consolation to people who spent 36 hours or more thinking they might be dead soon.
Careful selection and monitoring remains their responsibility and accountability typically remains with them. Delegation is not risk free although plenty of managers pretend otherwise.
I read the article, in my book the option of sending out a mass email of "you have cancer" should have been impossible. If a test showed that there is potential for cancer the email should have always been written by a real human. Instead it seems the "you have cancer" template sits in the same system with all the other "customer engagement" emails. If that's not a "sign of moving fast and break things" then I don't know. These different topics should have been compartmentalised into totally different departments that don't have anything to do with each other.
If you're sending thousands of emails with essentially identical content, it seems pointless to me to have them written by a human. Can you imagine the job of a human writing different wordings for "you have cancer" hundreds of times per day? Is that what we should be using humans for?
They have already explained the point. It's reducing the number of people who are incorrectly told they may soon die a horrible death.
If there are better ways to prevent that than a human-in-the-loop approach, feel free to propose it. But having watched it happen a couple of times, I'll note that actual doctors deliver cancer diagnoses in person, so it's not unreasonable to think that should continue to be the case.
There should be more emphasis on taking things slow and doing it right in the healthcare industry. I've also seen stuff that suggests overworked healthcare providers make more mistakes.
I would certainly agree with that for developing new diagnostic assays. But unfortunately clinicians doing direct patient care seldom have the luxury of taking things slow. Demand is growing every year and we aren't adding new providers at the same rate. Taking things slow with one patient means another patient has to wait in line.
That's extremely easy to say as someone with a serious medical condition who desperately wishes the TV show House were reality. I've read plenty of similar stories to mine.
I came to hate the 15 minute appointment with a passion.
On a system level, a false positive like this usually generates many, many follow-ups and additional tests just to be safe. That additional patient is now waiting significantly longer for the system to clear the backlog of high priority false alarms.
We can't ethically deny people care they actually need, so the most important tool we have to optimize healthcare resources is to avoid doing unnecessary things, even if it may seem slower upfront.
What would be the harm? It’s a blood test for a broad spectrum of cancers. So any doctor and patient should know that there is a non-zero false positive rate. So the harm could be CT scans, biopsies, colonoscopies, …
Also some kind of psychological trauma about the positive test result.
I would have liked to see the real error rates and how many tests they're selling.
It seems like this is not part of the false positive rate of the medical test itself. But it is part of the overall false positive rate of the organization.
The problem is the same as what destroyed the diagnosis through expert systems the first time around in the 80s/90s.. the software adds an asterisk behind every diagnosis, whether positive or negative, and says “..but talk to your doctor to be sure”. The consequence is that patients flood doctors, and you might as well remove the software from the equation.
Every test has some failure rate aka asterisk. A priori a we cannot dismiss a test for a high failure rate. It depends on how severe the condition is. So for really nasty cancers, we accept higher false positive rates because of the severe consequences.
I don’t know about these expert systems. Might very well be that they were overall useless.
Another scenario where it’s plausible is skin cancer. Let’s say there’s an app with a false positive rate of 10%. Now worried patients flood dermatologists. They quickly double check and send them home or treat them.
Don’t nail me down on numbers here. But it seems like an acceptable situation given that more and more people are dying of skin cancer.
I think you may be underestimating the importance of this. Say you have a test that detects skin cancer 60% of the time. ..flipping a coin would do it 50% of the time. Such a test is pretty useless. Remember that you’re applying this diagnosis to the broad population, not somebody who came to your practice with an initial suspicion.
There's a difference between diagnostic testing of individuals who have symptoms and population level screening.
Both involve statistics and probability, and the public (and a surprising number of health care professionals) really struggle with both.
To take your number: a disease exists. There's a test for that disease. The test is good, but not perfect. If you have the disease there is a 99% probability that the test will return "Positive". But if you do not have the disease there is a 10% chance that the test will also return "positive". 4 people in 100,000 people have the disease.
Hypothetical Bob takes the test. The test shows "positive". What is the probability that Bob actually has the disease?
Most of the public, and lots of healthcare providers, cannot even begin to answer this problem. They simply lack the math skill to start to work out what the answer is.
4 people out of 100,000 people have a disease. There is a test for the disease. If you have the disease the test will, 99 times out of 100, say "positive". If you do not have the disease the test will, 10 times out of 100, (100 out of 1,000, 1,000 out of 10,000 or 10,000 out of 100,000) say "negative". Bob has had the test. It said positive. What's the chances he has the disease?
Now people can see that only 4 people in 100,000 have the disease and most of those are going to get a positive result, but also 10,000 people who don't have the disease will also get a positive result.
We'd be overwhelming diagnostic services with people who do not have cancer because our test is terrible.
Of course, all of this changes if Bob has a mole that has changed shape, because the testing converts from a screening test to a diagnostic test.
Thanks for pointing it out so clearly. Bayesian statistics is the right tool for this analysis as we have a base rate in the population (prior) and a test accuracy (conditional probability).
Guess what CT scans cause (in aggregate populations): Cancer! It's literally a buttload of xrays.
To put some concrete numbers on it,
"They estimated that one full-body CT scan in a 45-year-old person confers an increased lifetime risk of cancer death of 0.08%, which is approximately 1 in 1250 people. Moreover, annual scans from ages 45 to 75 years could result in an increased risk of cancer mortality of 1.9%, or approximately 1 in 50 people."
Yes, annual is overkill more likely than not, but there are countries that do periodic CT scans above a certain age.
Knowing a number of people in their 30s-50s who passed or had close calls with slow moving but symptomless cancer caught early only because of an unrelated accident.. I'd think CT scans every 5 years or something past say 40ish probably has a good ROI for the individual. That said, it doesn't have a great ROI for insurance companies, which is why it doesn't happen.
There are numerous types of internal organ cancers that are incredibly slow moving (10-20+ years) but by the time they give you symptoms you are in stage 4. And many of these have essentially no alternative screenings for early warning - thyroid / liver / pancreas / kidney / etc.
Let me show you a more clear example: "sebzim4500, we have analysed your blood sample taken on the 2nd of June, 2023. The GM2345 general immuno-assay test indicates the presence of anti-globulins which tells us that you possibly have Hodgkin's lymphoma. We recommend that you start chemotherapy immediately. Please contact our office to arrange your treatment."
While it is true that you might have cancer (as you say a decent portion of the population does) but we know the above message is false. I do not have your blood sample, I did not run any tests, the name and description of the test is made up. All of these make the statement wrong. And in everyday vernacular someone might describe the above interaction as "sebzim4500 was wrongly told that he might have cancer."
"You might have cancer" is an example of a sentence whose meaning can't easily be analysed in terms of "truth conditions" (https://en.wikipedia.org/wiki/Truth_condition). When used appropriately, the speaker has information which they believe the listener does not already have and which, in the opinion of the speaker, should cause the listener to increase their estimate of the probabiliy of them having cancer ... or something like that. So the appropriateness of the statement depends rather crucially on the context and on prior knowledge. That's why sentences with the word "might" tend to become infuriatingly meaningless when quoted out of context by journalists.
I was told on the phone I had vitiligo cause the doc mixed up my results with those of another patient. Devastated upon getting the news, so much relief when I went to visit him to discuss treatment and he told me the mistake.
It sounds like you are hang up on the fact that everyone "might have cancer", thus technically everyone "might have cancer"?
It depends on the context of the sentence. It is unlikely that they just sent out a one line email saying "you might have cancer" and nothing more.
If the context of the letter makes it clear that they are raising awareness about the possibility of people having cancer (such as for example listing symptoms to look out for, or general population statistics) that cannot be done "wrongly". (It might be still unduly alarming, distasteful, factually incorrect or unethical for a host of other reasons.)
If the context is that they are telling you that they have run tests on your recent lab samples, and those tests are positive for the presence of cancer. That is an entirely different animal. That can be done "wrongly" if for example they have not done lab tests or the lab tests do not indicate that you have a higher probability of having cancer.
80% of men over 80 have prostate cancer. It is virtually an inevitability that if you live long enough, some cancers will develop. The questions which are more pertinent are: at what stage is your cancer, where is it located, is it metastasizing, and how progressive is it.
This test goes beyond "do you have cancer" by probabilistically identifying the presence, location, and progression of cancers. The results are shared with your doctor, who analyzes the results, gives them to you, and sets up your diagnostic and treatment plan.
Galleri does not talk directly to patients. This email was more like "your results say you need to follow up with your doctor" so the result was 400 people potentially having an extra doctor visit where they did not have any cancer results to discuss. Hardly malpractice.
Bill Burr put it correctly: "why would you send your saliva into the internet?"
Don't. You don't have any guarantees that these dubious companies will do right by your DNA. I'm surprised when people who wouldn't keep Alexa in their homes would send their DNA to ancestry.com. Like do you realise DNA is forever? You AND your entire bloodline is now forsaken genetic privacy for all eternity! A single leak and your genetic data is out in the internet for perpetuity!
I've even stopped giving blood samples to random pathology services and restrict to just one (no guarantees that they'll not fork me over either but at least minimise my exposure).
Sorry you have not researched this test enough. It identifies proteins in blood it does not sequence your genes, as cancer is not encoded in your genes.
Every time mistakes in cancer tests come up, I have to bring up this textbook counter-intuitive probabilistic result that happens when one tries to detect an a priori very rare condition.
It follows from a straightforward application of Bayes formula:
- Suppose a person has 1% chance of getting cancer, so p(cancer) = 0.01
- Suppose that the test has 99% sensitivity and specificity, i.e., p(positive | cancer) = p(negative | no cancer) = 0.99 (no test is perfect in the real world, in this case the test is wrong only one time every 100).
- To use Bayes we first need the probability of a positive result regardless of disease status, i.e. p(positive) = p(positive | cancer) * p(cancer) + p(positive | no cancer) * p(no cancer) = 0.99 * 0.01 + 0.01 * 0.99 = 0.0198
- Then by Bayes we have p(cancer | positive) = p(positive | cancer) * p(cancer) / p(positive) = 0.99 * 0.01 / 0.0198 = 0.5
And things get worse the rarer the condition is. For example, when p(cancer) = 0.001 then p(cancer | positive) = 0.09, while for p(cancer) = 0.5 the computations above give p(cancer | positive) = 0.99.
In other words, the rarer the condition, the more precise tests have to be to have confidence in its results.
If it's "set up correctly", then they'll only do a test like that if they have a reasonable follow-up test. In which case they will probably send a good and valid email and "this" is not what will happen.
In a sense this comment is still relevant, because from the grand perspective this email error could be considered a random failure on the part of the test in its full practical context. And it's not impossible it will happen again (in fact I expect it to). But it's certainly not characteristic of the test itself in theory, but is nevertheless characteristic of any such test that might involve bad IT. It's almost like test false-positive background radiation.
I first skimmed over "a priori" and thought you went to talking about prions; I don't think I could think about the statistics of those with a cool head much less cancer, however rare..
Essentially, this was a misconfigured transactional email campaign, and not the result of an erroneous test or false positive, which is an important, if a bit buried, lede here.
Why is "You might have cancer" even an email at all? Shouldn't that be done face to face?
When I was diagnosed with the cancer the nurses wouldn't even tell me I had cancer until I was sitting down with the haematologist, even so far as going to tell me when they asked me to come in to discuss my blood test that, "Don't worry that your appointment is in <St. Cancer wing>, that's just where our department is based". ( I didn't believe them, I was feeling incredibly rough at the time, cancer made sense. )
The letter didn't said "You have cancer, tell bye to your loved ones" with big bold letters. It said something like you should come to the hospital for further checks. Which in the internal procedure is done only for those that the test detected with cancer. (Source I'm in the pilot)
> Why is "You might have cancer" even an email at all? Shouldn't that be done face to face?
Not necessarily, but you’d have to phrase it as “we need to take a closer look”. If you’re doing population screening, you often have so many false positives that the probability that you have cancer, given that you had a positive test result, still is fairly low (for example, if 1:10,000 people have cancer, and the test is 100% reliable for those having cancer, and 99% for qthose who don’t, ≈1:100 people will be false positives, so in the group who tested positive, ≈99:100 would be false positives)
If you're asking if I have "beat" cancer, then no. My cancer isn't curable. It doesn't even technically go into "remission", although some nurses will refer to it as such once it gets to an undetectable level; others will be more precise about simply saying it's at an undetectable level. It's still there, it will return, most likely in a year or two.
I have already lived longer than the headline "statistics" would suggest, partly due to my unusual age (typical age is 80+, I'm less than half that), and partly due to new treatments that haven't yet fed into the historic survivability statistics.
I'm doing just fine for now though thanks, the biggest impact on my life is that I only work part time because I need time off for medical appointments and I'm unlikely to get a mortgage ( if I could get one, I wouldn't want to borrow from a bank irresponsible enough to lend to someone not predicted to live for half the mortgage term! ).
I'm fortunate to live somewhere that I've never so much as even seen a hospital bill, I've never had to worry about insurance or co-pays, or go fund-me. I've therefore had the privilege to focus entirely on my own health, as well as having supportive employers who have let me work flexibly throughout. I very much appreciate this doesn't apply to everyone, even within my own country if I worked in different circumstances or for different employers I may have had a very different experience.
I've veered very off topic from the original post though, so I'll probably rein in my posting about this topic now.
Doctors can misdiagnose and misunderstand or even just plain confuse patients too.
Grail (as I understand it) has the potential to completely revolutionise cancer treatment and while we shouldn’t go easy on them for a tech mistake like this we should get this technology rolled out and it might well save millions of lives. I bet the people who have now been told they don’t have cancer feel a great sense of relief and maybe a new desire to look after their health. Are we sure overall this sort of placebo shock won’t end up being a net positive for those involved?
Doctors have licenses, which are hard to obtain and can and are revoked for life for serious missteps. They need to have expensive insurance because patients sue them if they mess up, sometimes even if they don't. Doctors can end up in jail.
Even outside of US I can imagine it would be very easy to sue for emotional distress caused. Imagine somebody may quit their job on spot, do some financial decisions which they wouldn't do otherwise, or even commit suicide if already on the edge.
Without wanting to downplay the effects of being told you have cancer, one has to wonder if the doctor or lab should be responsible for whatever an unstable person does upon receiving this news.
The examples you mention would be just as bad even if the news was true. Not sure if there's any valid logic there on my part.
> Even outside of US I can imagine it would be very easy to sue for emotional distress caused
Living in the EU, I'm guessing it would have to be REALLY bad for you to be able to take someone to court over this. It would have to be "fake news" that persisted for a pretty long time, not just a phone call or email.
Put an innocent person in jail for X years, compensate them for it. Tell a person they're going to jail and tell them the next day that they aren't, that's a different story.
68 comments
[ 7.1 ms ] story [ 133 ms ] threadThat would probably invoke an "I'm being phished" reaction if I were in that situation, but I do wonder if the average person would realise the impossibility of the situation.
I would think the company is running a scam and sending random replies, astrology style, and that I got through a crack in their system.
First party is fully responsible for what vendors they choose to use. As a customer, I have little insight into that.
If there are better ways to prevent that than a human-in-the-loop approach, feel free to propose it. But having watched it happen a couple of times, I'll note that actual doctors deliver cancer diagnoses in person, so it's not unreasonable to think that should continue to be the case.
I came to hate the 15 minute appointment with a passion.
We can't ethically deny people care they actually need, so the most important tool we have to optimize healthcare resources is to avoid doing unnecessary things, even if it may seem slower upfront.
I would have liked to see the real error rates and how many tests they're selling.
It seems like this is not part of the false positive rate of the medical test itself. But it is part of the overall false positive rate of the organization.
I don’t know about these expert systems. Might very well be that they were overall useless.
Another scenario where it’s plausible is skin cancer. Let’s say there’s an app with a false positive rate of 10%. Now worried patients flood dermatologists. They quickly double check and send them home or treat them.
Don’t nail me down on numbers here. But it seems like an acceptable situation given that more and more people are dying of skin cancer.
Both involve statistics and probability, and the public (and a surprising number of health care professionals) really struggle with both.
To take your number: a disease exists. There's a test for that disease. The test is good, but not perfect. If you have the disease there is a 99% probability that the test will return "Positive". But if you do not have the disease there is a 10% chance that the test will also return "positive". 4 people in 100,000 people have the disease.
Hypothetical Bob takes the test. The test shows "positive". What is the probability that Bob actually has the disease?
Most of the public, and lots of healthcare providers, cannot even begin to answer this problem. They simply lack the math skill to start to work out what the answer is.
4 people out of 100,000 people have a disease. There is a test for the disease. If you have the disease the test will, 99 times out of 100, say "positive". If you do not have the disease the test will, 10 times out of 100, (100 out of 1,000, 1,000 out of 10,000 or 10,000 out of 100,000) say "negative". Bob has had the test. It said positive. What's the chances he has the disease?
Now people can see that only 4 people in 100,000 have the disease and most of those are going to get a positive result, but also 10,000 people who don't have the disease will also get a positive result.
We'd be overwhelming diagnostic services with people who do not have cancer because our test is terrible.
Of course, all of this changes if Bob has a mole that has changed shape, because the testing converts from a screening test to a diagnostic test.
Guess what CT scans cause (in aggregate populations): Cancer! It's literally a buttload of xrays.
To put some concrete numbers on it,
"They estimated that one full-body CT scan in a 45-year-old person confers an increased lifetime risk of cancer death of 0.08%, which is approximately 1 in 1250 people. Moreover, annual scans from ages 45 to 75 years could result in an increased risk of cancer mortality of 1.9%, or approximately 1 in 50 people."
Read More: https://www.ajronline.org/doi/10.2214/AJR.12.9226
Knowing a number of people in their 30s-50s who passed or had close calls with slow moving but symptomless cancer caught early only because of an unrelated accident.. I'd think CT scans every 5 years or something past say 40ish probably has a good ROI for the individual. That said, it doesn't have a great ROI for insurance companies, which is why it doesn't happen.
There are numerous types of internal organ cancers that are incredibly slow moving (10-20+ years) but by the time they give you symptoms you are in stage 4. And many of these have essentially no alternative screenings for early warning - thyroid / liver / pancreas / kidney / etc.
Just as an example of how you might wrongly tell someone they might have cancer
While it is true that you might have cancer (as you say a decent portion of the population does) but we know the above message is false. I do not have your blood sample, I did not run any tests, the name and description of the test is made up. All of these make the statement wrong. And in everyday vernacular someone might describe the above interaction as "sebzim4500 was wrongly told that he might have cancer."
Even further, cancer screening is not a replacement for cancer diagnosis.
It depends on the context of the sentence. It is unlikely that they just sent out a one line email saying "you might have cancer" and nothing more.
If the context of the letter makes it clear that they are raising awareness about the possibility of people having cancer (such as for example listing symptoms to look out for, or general population statistics) that cannot be done "wrongly". (It might be still unduly alarming, distasteful, factually incorrect or unethical for a host of other reasons.)
If the context is that they are telling you that they have run tests on your recent lab samples, and those tests are positive for the presence of cancer. That is an entirely different animal. That can be done "wrongly" if for example they have not done lab tests or the lab tests do not indicate that you have a higher probability of having cancer.
This test goes beyond "do you have cancer" by probabilistically identifying the presence, location, and progression of cancers. The results are shared with your doctor, who analyzes the results, gives them to you, and sets up your diagnostic and treatment plan.
Galleri does not talk directly to patients. This email was more like "your results say you need to follow up with your doctor" so the result was 400 people potentially having an extra doctor visit where they did not have any cancer results to discuss. Hardly malpractice.
Don't. You don't have any guarantees that these dubious companies will do right by your DNA. I'm surprised when people who wouldn't keep Alexa in their homes would send their DNA to ancestry.com. Like do you realise DNA is forever? You AND your entire bloodline is now forsaken genetic privacy for all eternity! A single leak and your genetic data is out in the internet for perpetuity!
I've even stopped giving blood samples to random pathology services and restrict to just one (no guarantees that they'll not fork me over either but at least minimise my exposure).
It follows from a straightforward application of Bayes formula:
- Suppose a person has 1% chance of getting cancer, so p(cancer) = 0.01
- Suppose that the test has 99% sensitivity and specificity, i.e., p(positive | cancer) = p(negative | no cancer) = 0.99 (no test is perfect in the real world, in this case the test is wrong only one time every 100).
- To use Bayes we first need the probability of a positive result regardless of disease status, i.e. p(positive) = p(positive | cancer) * p(cancer) + p(positive | no cancer) * p(no cancer) = 0.99 * 0.01 + 0.01 * 0.99 = 0.0198
- Then by Bayes we have p(cancer | positive) = p(positive | cancer) * p(cancer) / p(positive) = 0.99 * 0.01 / 0.0198 = 0.5
And things get worse the rarer the condition is. For example, when p(cancer) = 0.001 then p(cancer | positive) = 0.09, while for p(cancer) = 0.5 the computations above give p(cancer | positive) = 0.99.
In other words, the rarer the condition, the more precise tests have to be to have confidence in its results.
It‘s a misleading title for sure.
When I was diagnosed with the cancer the nurses wouldn't even tell me I had cancer until I was sitting down with the haematologist, even so far as going to tell me when they asked me to come in to discuss my blood test that, "Don't worry that your appointment is in <St. Cancer wing>, that's just where our department is based". ( I didn't believe them, I was feeling incredibly rough at the time, cancer made sense. )
Not necessarily, but you’d have to phrase it as “we need to take a closer look”. If you’re doing population screening, you often have so many false positives that the probability that you have cancer, given that you had a positive test result, still is fairly low (for example, if 1:10,000 people have cancer, and the test is 100% reliable for those having cancer, and 99% for qthose who don’t, ≈1:100 people will be false positives, so in the group who tested positive, ≈99:100 would be false positives)
If you're asking if I have "beat" cancer, then no. My cancer isn't curable. It doesn't even technically go into "remission", although some nurses will refer to it as such once it gets to an undetectable level; others will be more precise about simply saying it's at an undetectable level. It's still there, it will return, most likely in a year or two.
I have already lived longer than the headline "statistics" would suggest, partly due to my unusual age (typical age is 80+, I'm less than half that), and partly due to new treatments that haven't yet fed into the historic survivability statistics.
I'm doing just fine for now though thanks, the biggest impact on my life is that I only work part time because I need time off for medical appointments and I'm unlikely to get a mortgage ( if I could get one, I wouldn't want to borrow from a bank irresponsible enough to lend to someone not predicted to live for half the mortgage term! ).
I'm fortunate to live somewhere that I've never so much as even seen a hospital bill, I've never had to worry about insurance or co-pays, or go fund-me. I've therefore had the privilege to focus entirely on my own health, as well as having supportive employers who have let me work flexibly throughout. I very much appreciate this doesn't apply to everyone, even within my own country if I worked in different circumstances or for different employers I may have had a very different experience.
I've veered very off topic from the original post though, so I'll probably rein in my posting about this topic now.
Grail (as I understand it) has the potential to completely revolutionise cancer treatment and while we shouldn’t go easy on them for a tech mistake like this we should get this technology rolled out and it might well save millions of lives. I bet the people who have now been told they don’t have cancer feel a great sense of relief and maybe a new desire to look after their health. Are we sure overall this sort of placebo shock won’t end up being a net positive for those involved?
Even outside of US I can imagine it would be very easy to sue for emotional distress caused. Imagine somebody may quit their job on spot, do some financial decisions which they wouldn't do otherwise, or even commit suicide if already on the edge.
The examples you mention would be just as bad even if the news was true. Not sure if there's any valid logic there on my part.
> Even outside of US I can imagine it would be very easy to sue for emotional distress caused
Living in the EU, I'm guessing it would have to be REALLY bad for you to be able to take someone to court over this. It would have to be "fake news" that persisted for a pretty long time, not just a phone call or email.
Put an innocent person in jail for X years, compensate them for it. Tell a person they're going to jail and tell them the next day that they aren't, that's a different story.
While this is unfortunate, false positives do happen.