According to the CDC [0], the five-year net survival rate for localized pancreatic cancer is 37.4%. That's really low for an early-staged diagnosis. By comparison, early-staged colorectal cancer has an 89.9% net survival rate, despite the late-staged rate being 14.2%.
Granted, a small proportion of these cancers are caught an early stage. But I wouldn't say most people diagnosed would live five years longer if they'd caught it early. And that's ignoring lead-time bias [1].
This depends on the type of cancer. In the case of a slow-growing cancer like prostate cancer, early detection can find cancers that would never be a problem (you'll die of something else first).
This has to be taken into account when deciding how much screening to do.
Not really. You’re probably still better off knowing you have the slow-growing benign prostrate cancer that you can monitor but not treat. Also, you might find you have the more aggressive form.
Maybe the problem is that people don't do that? What do you think of this meta-analysis?
"Pooled data currently demonstrates no significant reduction in prostate cancer-specific and overall mortality. Harms associated with PSA-based screening and subsequent diagnostic evaluations are frequent, and moderate in severity. Overdiagnosis and overtreatment are common and are associated with treatment-related harms."
[...]
"Screening resulted in a range of harms that can be considered minor to major in severity and duration. Common minor harms from screening include bleeding, bruising and short-term anxiety. Common major harms include overdiagnosis and overtreatment, including infection, blood loss requiring transfusion, pneumonia, erectile dysfunction, and incontinence. Harms of screening included false-positive results for the PSA test and overdiagnosis (up to 50% in the ERSPC study). Adverse events associated with transrectal ultrasound (TRUS)-guided biopsies included infection, bleeding and pain."
many people also insist on getting antibiotics when they have a viral infection. We have a more measles cases now than a decade ago because of fake news.
You might be right that many people would be better off not knowing but I’d rather we took a step forwards than backwards.
Early detection of breast cancer has had little noticeable impact on the mortality rates of breast cancer. Instead, it's been observed that the improved treatments for all forms of cancer have been the cause of a decrease in breast cancer mortality rates.
Without information about false positives, this is just basically saying they wrote an algorithm that sometimes points out cancer early. But if it is only correct 1% of the time, nobody is going to listen to it. It'd do even less than the current "You really need to check for cancer!" statements that we already have.
Edit: From the paper:
> A deep learning (DL) mammography-based model identified women at high risk for breast cancer and placed 31% of all patients with future breast cancer in the top risk decile compared with only 18% by the Tyrer-Cuzick model (version 8).
So better than before, but still only detects 31%. If I'm reading correctly, it's 95% correct? I guess that means 5% false positives? That wouldn't be bad.
You can be told the risk factors for everybody, but when you are told the risk factors tailored to you it becomes dangerous? Should we tell everyone they're going to live forever so they live in ignorant bliss while healthy?
I can see being unprepared for having that information, but I don't think the solution is not having the information.
Specifically on breast cancer screening, the National Institute for Clinical Evidence in the UK has published management guidance that is quite interesting: https://cks.nice.org.uk/breast-screening
They wouldn't be wold they were positive for cancer. They'd be told that the computer predicts that they are at very high risk for breast cancer. That's knowledge that I think people should have, personally.
Sorry, please explain to me if my interpretation is correct:
1. The cost of a false positive is low.
1a. The cost of a false positive is retesting (what is retesting? Re-mammogramming? Isn't healthcare in USA expensive?)
1b. Observe (so future re-mammogramming and hypervigilance).
2. Additional to the cost of 1a and 1b, cease consuming sugar. Which is a non-insignificant lifestyle change.
What I'm saying is... the statements "Rerun expensive tests and maintain hypervigilance seems like a low cost, in addition to major lifestyle changes that affect every level of one's socioeconomic place (eating out with friends/family, cost of food, food availability, habit breaking, possibly culture clashes, etc)." seems pretty much equivalent to absolute nonsense.
Given a positive result. P of having the disease are 2%. Given a false result, it’s about 0. Given 2 positive results, assuming independent results (I think that’s unlikely but let’s assume) then the probability of having it is 0.04.
I’m not disagreeing that there usefulness to tests, but there is danger too.
It seems they are simply assigning patients to the top risk decile (more accurately than before). Telling someone they are at a high risk is different than a flat-out diagnosis. Better efficacy of the predictive model (fewer false positives and fewer false negatives than before) is a good thing.
> I guess that means 5% false positives? That wouldn't be bad.
Due to Bayes' Theorem, this can result in the majority of patients with a positive result being patients without cancer. Depending on what treatment is taken, and the psychological burden of erroneously thinking you have cancer, the negatives of the additional screening could outweigh the positives.
FWIW, it's not like the doctor is going to send you home with a diagnosis based on this screening. The psychological burden would persist only until further testing and analysis. And in any case, human radiologists have FPs and FNs, too. The question is how close and in what circumstances are we approaching current human radiologist performance.
No, the problem is work up rate. If you significantly increase the rate of biopsy (typically one of the possible next steps) eventually you are going to kill people, as biopsy is not a zero risk procedure.
You have to be really careful with FP rates in screening. Of course the answer to this could just be "don't biopsy due to only this result" but at some point that renders the approach pointless, if you can't find a cost effective way of mitigating the FPR.
TLDR but 95% correct does not necessarily equate to 5% false positives: you can also have false negatives. The bottom line is the new algorithm increases the efficacy of the prediction.
But it said it only detects future cancer in 31% of patients, which means way more than 5% false negatives. I'm assuming that means that the 95% is only on the positives.
Addressing model bias by adjusting which data the model has access to is a bad idea. Tweaking the data so that the model output looks equitable is going to make your model across the board. You should train your model on what you have and then add explicit biases to the classifier for different groups. That way you have the best model and are clear on your biases.
If this model is equally accurate for black and white women that means that either race is not a factor in predictability, that it is a factor but easily adaptable into a model, or race is a factor and they’ve reduced their ability to diagnose one group in the name of equity.
The linked article suggests accuracy gains are due to better risk models, that use more than age. I’m not sure if that means it’s tied into the image neural net. Would like to see false positive rate too.
I wish they would call it something other than AI. Like "Diagnostics" or if there MUST be a buzzword there, then call it "Predictive Diagnostics".
Once upon a time, a necessary precondition to call something AI was that it should be something where there is at least the hope that it could one day generalize to pass the Turing test or something along those lines.
Medical diagnostics is one of the primary applications of pattern processing, and since it's pretty damned impressive as it is, it's a bit pointless to try and make it even more impressive by suggesting that you might one day enjoy a chat with your medical diagnostic tool over breakfast, exchanging views on how the Knicks' season is shaping up... (Which both the informed readers, and the people writing this, know pretty damned well is never going to happen, and was never intended to happen).
Take these results with a grain of salt. There's a large class imbalance in this dataset and ROC curves can be misleading in this case. The test set contains 269 positive examples and 8482 negative examples.
From [1]:
> Class imbalance can cause ROC curves to be poor visualiza-
tions of classifier performance. For instance, if only 5 out of 100
individuals have the disease, then we would expect the five posi-
tive cases to have scores close to the top of our list. If our classifier
generates scores that rank these 5 cases as uniformly distributed
in the top 15, the ROC graph will look good (Fig. 4a). However,
if we had used a threshold such that the top 15 were predicted to
be true, 10 of them would be FPs, which is not reflected in the
ROC curve. This poor performance is reflected in the PR curve,
however.
The authors seem to be aware of this in the supplement and also evaluate performance by a hazard ratio they define:
> We calculated the ratio of the observed cancer incidence in the top 10% of patients over the incidence in the middle 80% and referred to this metric as the top decile hazard ratio. We calculated the ratio of the observed cancer incidence in the bottom 10% of patients over the incidence in the middle 80% and referred to this metric as the bottom decile hazard ratio.
However, binning is a form of p-hacking [2]. And I'm still wondering why they don't just post the Precision-Recall curves.
It's one of the difficulties with attacking screening applications where the population TPR is very low. For screening mammo, it's less than 1 in 1000.
.. TESTIMONY OF HOW MY DAD GOT CURED FROM CANCER......
My name is Brown, my father has been suffering from cancer for over 3years, and the doctor told us that there is nothing he could do after doing 5 times chemotherapy and radiation and my father was not responding to treatment and not getting better.i was frustrated and confused and i started seeking advice from people until a friend told me about Dr. ALABI herbal medicines and cannabis oil and its fast curing ability of any kind of cancer and told me that her mother ovarian cancer was cured with the cannabis oil. I was desperate to save my father and i contacted Dr. ALABI via his email adress: dralabim5@gmail.com to get the miraculous oil. that was how the oil was sent to me through UPS Currier service. within 4days As i am writing this testimony on this Blog my father is so strong and healthy no more cancer in his cell blood again.my Father has been himself and i am so exicted of his fast recovery and with the help of this cannabis oil. What a miracle. My Father is now back alive, strong and healthy just within 3 weeks. We have gone for test and the doctor said there is no trace of cancer again in his body. you can also reach him via his whatsapp number:+2348155011610...Dr, ALABI CAN AS WELL CURE THE FOLLOWING DISEASES:
1. HIV/AIDS
2. HERPES
3. CANCER
4. ALS
5. Hepatitis B
6. Diabetes
THANK YOU VERY MUCH FOR TAKING YOUR TIME TO READ THROUGH MY TESTIMONY
33 comments
[ 4.4 ms ] story [ 82.2 ms ] threadhttps://youtu.be/iUqgTYbkHP8?t=15m37s
The reason most people die from pancreatic cancer, for example, is because we almost always detect it in a late stage.
Granted, a small proportion of these cancers are caught an early stage. But I wouldn't say most people diagnosed would live five years longer if they'd caught it early. And that's ignoring lead-time bias [1].
[0] https://seer.cancer.gov/statfacts/html/pancreas.html
[1] https://en.m.wikipedia.org/wiki/Lead_time_bias
This has to be taken into account when deciding how much screening to do.
https://www.webmd.com/prostate-cancer/prostate-cancer-surviv...
"Pooled data currently demonstrates no significant reduction in prostate cancer-specific and overall mortality. Harms associated with PSA-based screening and subsequent diagnostic evaluations are frequent, and moderate in severity. Overdiagnosis and overtreatment are common and are associated with treatment-related harms."
[...]
"Screening resulted in a range of harms that can be considered minor to major in severity and duration. Common minor harms from screening include bleeding, bruising and short-term anxiety. Common major harms include overdiagnosis and overtreatment, including infection, blood loss requiring transfusion, pneumonia, erectile dysfunction, and incontinence. Harms of screening included false-positive results for the PSA test and overdiagnosis (up to 50% in the ERSPC study). Adverse events associated with transrectal ultrasound (TRUS)-guided biopsies included infection, bleeding and pain."
https://www.cochrane.org/CD004720/PROSTATE_screening-for-pro...
You might be right that many people would be better off not knowing but I’d rather we took a step forwards than backwards.
https://www.sciencedirect.com/science/article/pii/S221353831...
https://pubs.rsna.org/doi/full/10.1148/radiol.2019182716
Edit: From the paper:
> A deep learning (DL) mammography-based model identified women at high risk for breast cancer and placed 31% of all patients with future breast cancer in the top risk decile compared with only 18% by the Tyrer-Cuzick model (version 8).
So better than before, but still only detects 31%. If I'm reading correctly, it's 95% correct? I guess that means 5% false positives? That wouldn't be bad.
950 true positives
~50,000 false positives
False positive breast cancer is pretty bad.
You can be told the risk factors for everybody, but when you are told the risk factors tailored to you it becomes dangerous? Should we tell everyone they're going to live forever so they live in ignorant bliss while healthy?
I can see being unprepared for having that information, but I don't think the solution is not having the information.
There have been studies into exactly this: https://www.ncbi.nlm.nih.gov/pubmed/22859786
Specifically on breast cancer screening, the National Institute for Clinical Evidence in the UK has published management guidance that is quite interesting: https://cks.nice.org.uk/breast-screening
They wouldn't be wold they were positive for cancer. They'd be told that the computer predicts that they are at very high risk for breast cancer. That's knowledge that I think people should have, personally.
1. The cost of a false positive is low. 1a. The cost of a false positive is retesting (what is retesting? Re-mammogramming? Isn't healthcare in USA expensive?) 1b. Observe (so future re-mammogramming and hypervigilance).
2. Additional to the cost of 1a and 1b, cease consuming sugar. Which is a non-insignificant lifestyle change.
What I'm saying is... the statements "Rerun expensive tests and maintain hypervigilance seems like a low cost, in addition to major lifestyle changes that affect every level of one's socioeconomic place (eating out with friends/family, cost of food, food availability, habit breaking, possibly culture clashes, etc)." seems pretty much equivalent to absolute nonsense.
I’m not disagreeing that there usefulness to tests, but there is danger too.
Due to Bayes' Theorem, this can result in the majority of patients with a positive result being patients without cancer. Depending on what treatment is taken, and the psychological burden of erroneously thinking you have cancer, the negatives of the additional screening could outweigh the positives.
You have to be really careful with FP rates in screening. Of course the answer to this could just be "don't biopsy due to only this result" but at some point that renders the approach pointless, if you can't find a cost effective way of mitigating the FPR.
If this model is equally accurate for black and white women that means that either race is not a factor in predictability, that it is a factor but easily adaptable into a model, or race is a factor and they’ve reduced their ability to diagnose one group in the name of equity.
The linked article suggests accuracy gains are due to better risk models, that use more than age. I’m not sure if that means it’s tied into the image neural net. Would like to see false positive rate too.
Once upon a time, a necessary precondition to call something AI was that it should be something where there is at least the hope that it could one day generalize to pass the Turing test or something along those lines.
Medical diagnostics is one of the primary applications of pattern processing, and since it's pretty damned impressive as it is, it's a bit pointless to try and make it even more impressive by suggesting that you might one day enjoy a chat with your medical diagnostic tool over breakfast, exchanging views on how the Knicks' season is shaping up... (Which both the informed readers, and the people writing this, know pretty damned well is never going to happen, and was never intended to happen).
From [1]:
> Class imbalance can cause ROC curves to be poor visualiza- tions of classifier performance. For instance, if only 5 out of 100 individuals have the disease, then we would expect the five posi- tive cases to have scores close to the top of our list. If our classifier generates scores that rank these 5 cases as uniformly distributed in the top 15, the ROC graph will look good (Fig. 4a). However, if we had used a threshold such that the top 15 were predicted to be true, 10 of them would be FPs, which is not reflected in the ROC curve. This poor performance is reflected in the PR curve, however.
The authors seem to be aware of this in the supplement and also evaluate performance by a hazard ratio they define:
> We calculated the ratio of the observed cancer incidence in the top 10% of patients over the incidence in the middle 80% and referred to this metric as the top decile hazard ratio. We calculated the ratio of the observed cancer incidence in the bottom 10% of patients over the incidence in the middle 80% and referred to this metric as the bottom decile hazard ratio.
However, binning is a form of p-hacking [2]. And I'm still wondering why they don't just post the Precision-Recall curves.
[1] https://doi.org/10.1038/nmeth.3945
[2] https://doi.org/10.1080/09332480.2006.10722771
[Edit] to add link to [2]