... and the $500+ dermascope that is also required.
The combined items (phone+scope) probably do cost way less than whatever specialized systems are in place at most dermatologists (since the cost of most medical devices are obscene), but probably also come nowhere close to the price that could be achieved by talented hardware hackers who really wanted to get the price as low as possible.
Did you read the article? It also requires a $500 camera attachment device. It seems to be aimed more at local doctors than regular consumers and has an accuracy rate higher than the average primary care physician accuracy rate.
The technique is reported to be 85% accurate. That is, there's 15% chance of false positive. According to wikipedia (http://en.wikipedia.org/wiki/Melanoma#Epidemiology), upper bound for melanoma incidence is 20 per 100000 inhabitants. Consider that 100000 people used the technique to screen themselves. There would be 15000 of them that would got the positive test result and only 20 of them who have melanoma. Thus, for a person having the positive result the probability that he indeed has the melanoma is LESS than 0.013%. That is , the probability of that the technique is reporting B.S. is over 99.8%.
The fatal flaw in your reasoning lies in your assumption that the device will be used to screen for melanoma in the entire population. In reality, it would be used only on patients who show other signs that would normally result in a referral to a specialist (i.e., the chance they have melanoma is much higher than 20/100000). The likelihood of an accurate diagnosis is 85% when this population is examined, and not the general population. As the article states, the likelihood of an accurate diagnosis (85%) is the same regardless of whether it's performed in the traditional manner by a specialist, or if it's performed using this device by a physician who would have otherwise referred the patient to a specialist.
A great statistical point--but assuming they're marketing this to (local/small town) doctors rather than laypeople, is that worse than they'd do by "eyeballing" it?
There's not enough information to know what "85% accuracy" means. Is that false positive, or false negatives, or a combination of both, or misreporting?
I agree that presenting risks as percentages instead of real numbers is misleading.
But they say this device is better than primary care doctors so I'm not sure what that says about primary care.
Gerd Gigerenzer wrote a book in 2002 about clinicians inability to understand percentages and screening tests.
I'm not sure that's how it works when it comes to accuracy of medical screening devices.
Say if there is a device that offers 99.9% accuracy but is used to screen a disease that occurs 1 in 100,000, then applying your logic would mean out of the 100 people who were identified positive, only 1 actually was accurate, thus makes the actual result of the device 99% false.
The 85% here is likely to refer to that out of the positive results, there is a 15% false-positive rate.
Medical screening device accuracies aren't calibrated to how rare the actual disease is.
It's not about rarity of the disease. The problem with those percentages is that they're commercial bragging. The 99.9% accuracy is indeed very poor accuracy in case where there is 99.999% chance that one doesn't have a disease at all. It's just measuring the length of a bacteria with carpenter's tape ruler - the tool is too poor.
I could be wrong, but it's not the false positives you need to worry about - it's the false negatives.
15% false positive is no big deal. You can get confirmation tests, biopsies, second opinions, etc etc. But if someone scans a cancerous mole and is told "nope, all clear"? That's worrisome.
Single-image analysis is never going to be that accurate unless you've got a melanoma which already is very advanced. Detecting a changing lesion is a much more effective way of spotting a potential melanoma, and can catch something much sooner than these apps, many of which have come before.
Can you comment on the dermoscope? I'm curious why something like this costs 500$ - it sounds like a plastic lens and some LEDs would fit the job description.
A dermascope uses different wavelengths of light to get a picture of what's going on slightly below the skin as well as on its surface. This requires special lenses and lights, which I imagine are not used outside of this specific case, hence the cost.
I wonder if the iPhone 5s fingerprint scanner could be hacked to produce a rudimentary system it is meant to to involve 'subdermal scanning' according to the Apple marketing department.
20 comments
[ 0.29 ms ] story [ 45.9 ms ] threadThe combined items (phone+scope) probably do cost way less than whatever specialized systems are in place at most dermatologists (since the cost of most medical devices are obscene), but probably also come nowhere close to the price that could be achieved by talented hardware hackers who really wanted to get the price as low as possible.
Influenced by recent HN post: https://www.sciencenews.org/blog/context/doctors-flunk-quiz-...
TL;DR: 73.6% of all statistics are made up.
I agree that presenting risks as percentages instead of real numbers is misleading.
But they say this device is better than primary care doctors so I'm not sure what that says about primary care.
Gerd Gigerenzer wrote a book in 2002 about clinicians inability to understand percentages and screening tests.
http://www.amazon.co.uk/gp/aw/d/0140297863?pc_redir=13991465...
http://plus.maths.org/content/reckoning-risk
http://www.amazon.com/gp/aw/d/0140297863?pc_redir=1399339535...
Only that primary care providers are not specialists.
But your point is important: is using this device as a gateway to dermatologist referrals a good thing.
[0] https://www.youtube.com/watch?v=UF1T7KzRnrs&list=UUabaQPYxxK...
Say if there is a device that offers 99.9% accuracy but is used to screen a disease that occurs 1 in 100,000, then applying your logic would mean out of the 100 people who were identified positive, only 1 actually was accurate, thus makes the actual result of the device 99% false.
The 85% here is likely to refer to that out of the positive results, there is a 15% false-positive rate.
Medical screening device accuracies aren't calibrated to how rare the actual disease is.
15% false positive is no big deal. You can get confirmation tests, biopsies, second opinions, etc etc. But if someone scans a cancerous mole and is told "nope, all clear"? That's worrisome.
Disclosure: I work for Skin Analytics http://skin-analytics.com