While I think IBM Watson will become a powerful tool for querying and hospital/insurance risk management, I'm not sure it will be replacing a doctor until MRI machines are a dime a dozen. What good is a medical query engine if the patient doesn't know how to describe what is wrong? If you are planning your child's career, take this into account; otherwise, it is just something new for the doctors to learn, like engineers learning CAD tools.
I've seem papers before where machine learning was able to compensate for both inaccuracy of measurement and outright falsehoods being reported, and overall was far more reliable than humans.
Full Disclosure: I just read a paper saying that only 10% of doctors are capable of bayesian reasoning, so I'm in a mood to pick on doctors today. I'll stop now.
How big is a market for an expensive senior oncologist, if a doctor with twice shorter training (and half the school debt) can give the same results (by using the same Watson) while competing on price and friendliness?
"What good is a medical query engine if the patient doesn't know how to describe what is wrong?"
This implementation of Watson is more than a query engine; it returns a confidence interval with its diagnoses and "knows" which questions to ask to improve the confidence level in those diagnoses.
It's hard to take seriously the opinions of someone who knows nothing about AI and is therefore unable to critically evaluate the made about Watson. To go from any kind of parsing/understanding of medical articles, to actual diagnoses of patients, is absolutely impossible with today's technology
Going from symptoms and test results to a diagnosis, and doing it more accurately than humans is certainly possible with current technology. It's not a particular testament to AI quality - it's about the data quantity and the well-known inabilities of humans to accurately judge weights of evidence; humans are really good for effective judgements about common patterns and corelations, however, people are really really horrible at judgement when a multitude of individually rare possibilities are involved, their biases overwhelm the signal.
I'm not sure what you're talking about. I know something about AI. A couple of years ago when Watson destroyed the best human Jeopardy champions, I immediately saw its near-term applicability in sifting through the enormous space of medical data/research and correlating it with patient information to yield amazingly accurate and insightful diagnoses.
This isn't after-the-singularity type science fiction where we're adding assumption upon assumption to several levels beyond what is possible.
This is applying a technology (that's already proven at performing similar complex deductive analysis over massive quantities of data) to a new discipline. Any assumptions about what we can do in the new discipline with the current technology are minimal and extremely likely overcomable.
Even if Watson isn't quite there, the way is pretty clear now. Linear improvements in the Watson approach will yield results.
I would not be surprised if Watson is getting more value from the patient records than the journal articles. Even if patient records are unstructured, they typically are more likely to have key details of import for making rare subtle diagnoses that are not present in journal articles.
I used to think that until I dated a medical student. It is heartbreaking to me and I'd grasp at any reason or bit of information to be more optimistic but honestly, get real... How do you think a Venn diagram for medical and computer science (and/or statistics) students looks like?
Unlike with jeopardy, you do not have a huge well organized catalog of information lovingly curated by trivia worshiping nerds. It is a ridicules stumbling block but there you have it. Health records are not normalized and you can't touch them anyway, research is closer to human readable prose and filled with jargon - good luck developing NLP for it if you're "just a programmer". Hell, look at their textbooks and the way they study and are trained, that screams for computerized improvement and is still a tall order.
Actually there are already systems in the field that offer better results than people. I remember reading research about decision support systems doing a better job, and there's isabel healthcare , offering decision support system that works great for rare diseases as far as i know.
The way they are implemented is by people entering data in an orderly fashion, both those who built the systems ,and doctors . This somewhat limits the system, both from breadth of research and data entry time at the doctor.
But even with those limitations , those systems offer far better diagnoses , especially in the rare cases,than you average and even great doctor.
You didn't "have a huge well organized catalog of information lovingly curated", however, as the article states, IBM has been working on curating that information for many years already, and they claim that they got all of the knowledge that's taught to med-students already two years ago. So now they can go for the research papers to ensure that the system knows far more than a doctor could read in a lifetime.
You misunderstand how Watson was created. A bunch of trivia worshiping nerds didn't organize and catalog all of its information in the way you're implying. Watson doesn't need perfectly digested raw materials to work with.
Watson is able to parse natural language and pull out possibly important information. It then applies some very state-of-the-art machine learning to derive the most likely contextual meaning of that information. That contextual meaning can be cross-referenced with billions of other pieces of weighted information to derive further insights.
I come from a family full of doctors. They tend to be smart people, but as Jeopardy showed, even the best player on earth is no match for a machine trained at that discipline.
Once we start adding in genetic information, that will allow machines like Watson to customize diagnoses and treatments based upon your specific genetic profile.
On the parsing/understanding side, the main barrier is that it is only possible to infer very simple relationships. Either collecting facts with some simple schema (like "is a" relationships), or inferring mood/topics like "this article seems to be about cancer".
On the machine learning side, there is very little in the way of inference. Most of machine learning is an attempt to solve the statistical problem of predicting some value Y (e.g. a diagnosis) given a feature vector X.
To obtain data from medical texts (even with human assistance) and use this to form diagnoses of actual patients, would require solving both these problems. We would have to parse and represent the complex logical relationships that are expressed in academic papers. We would then have to have machine learning methods that can work with complex data (not just feature vectors) and combine inference with statistical learning.
Didn't the jeopardy win hinted that watson has inference capabilities ?
And isn't doctors and insurance companies being impressed with results are another hint that watson has those capabilities(or is a very big fraud, something IBM wouldn't riskm) ?
As with other technical advances, it’s unlikely to cause direct job loss, and more likely to make the existing workforce more productive and the product better and cheaper.
It might, at the margins, make ‘miracle-worker’ doctors less so. Those doctors will be less able to demand exorbitant fees.
It will also make more-accurate cancer diagnosis available to more doctors, and therefore to more patients. But no one (in the medium term) will not use an oncologist because Watson exists.
These are all good things. In finance, the computers are doing most of the work but employment hasn’t waned (at least not because of the computers).
The underlying reason is that demand for improved results, in both fields, has no foreseeable upper bound. Demand grows to consume the new capacity. There is no upper limit on ‘better cancer diagnosis’, as there is no upper limit on ‘better financial returns’.
Put another way, Watson-assisted diagnosis should be thought of as a ‘new’ product, stoking new demand, in the same way better Macs drive demand every year.
Someday you’ll buy a cheap health robot at Walmart, but this development is not that.
(ps Tyler Cowen’s new book has some interesting stuff on this topic.)
My mom is suffering from end-stage cancer and my experience with modern cancer medicine is that it is farce compared to the things we do in our profession in the name of getting someone to click on a banner ad. Given the rate at which this is progressing, I'd be more than happy to let Watson handle my care within 3-4 years time. The point at which something like Watson can wholly replace doctors is much closer than you think if these claims are truly valid and not being excessively pumped up for PR purposes.
Don't get me wrong, her doctor is a wonderful person that is very knowledgeable, but the way cancer medicine is done today is a lot of guess work and experimentation, with long lead times between measurements (blood tests and whatnot) that are hardly statistically valid and uber prone to tampering (used loosely in the quality control sense).
The existing workforce that will become far more productive are not the doctors. It's the nurses and physician's assistants. They provide value that is much harder to automate, although the Japanese certainly are trying to make progress in this area with humanoid robots to care for patients.
the way cancer medicine is done today is a lot of guess work and experimentation
On top of the things you point out, cancer is all about the genes - the genes in the host as well as the myriad of mutations in the cancer.
Cancer treatment of the future will involve sequencing of the patient's genome and repeated sequencing of samples of the cancer itself. The computer will be needed to dynamically account for mutations and prescribe treatment for each one as it arises. Networks of medical computer systems will be able to consult at internet speeds on specific mutations to get suggestions for treatment that are built upon results of other computer prescriptions whose efficacy is much better recorded, processed, and correlated.
As you say, it will be the second tier of the medical industry that most benefits from this revolution. Any nurse who can follow the instructions given by the computer for gathering information, administering tests, feeding results back to the computer, and providing treatment could become a world class oncologist.
Probably doctors will still play a role in some level of oversight of what the computers are prescribing -- but that will fade as our trust level increases over time. Within ten years, computers will be assisting doctors on every diagnosis. Within twenty, they'll be assisting nurses, mostly replacing the need for doctor time. Within thirty years, you'll be able to perform your own interactions with a medical computer as easily as you Google today -- probably easier since it will interact with you using natural language.
The opening chapters of Anne Macaffrey's, "The Ship a Who Searched" show the problems of relying on a cheap home healthcare robot. Hypatia Cade might not have become a brain ship if the robot had been able to tell that Tia was not just engaging in attention-seeking behaviour.
This has a very interesting secondary use: find studies or facts that most effect the diagnostics or prescriptions. Studies that are most relied on would benefit from replication attempts.
> find studies or facts that most effect the diagnostics or prescriptions
You could also expand that idea to having it acting as a spam filter for medical publications. It could scan every old and new published paper for basic validity. This wouldn't prevent fraud, but it might raise the bar for publishing bad science.
People who don't understand either artificial intelligence or medicine, or neither, tend to overlook the fine point that the particular tasks AI is suited for in medicine generally are those tasks the doctors like the least.
Doctors might like reading research every once in a while. They don't like spending all their time on it. An overworked doctor should not care if he refers a patient to an overworked oncologist or asks watson first.
Computer vision applied in health care also automates tasks that are mind-numbing. Is it really necessary for highly-skilled doctors with decades of experience to look through a microscope?
Doctors also complain that they don't have enough time for their patients. Watson seems to be excellent at completing paperwork. Given that we hardly have enough doctors anywhere in the world, more AI will be good thing.
I don't believe this is true. there was an attempt to add AI(sort of) to medicine in the form of decision support tools , but doctors didn't like them because it took autonomy from their job and made it boring: instead of figuring all kinds of stuff, suddenly doctors are being told what do by a machine.
It would be even worse psychologically - if some machine comes up and makes better diagnoses, then it means a multitude of cases where the doctor will violently disagree with the machine, (s)he will know what's right for the patient.. and still be wrong in most, but not all the cases, so listening to the doctor would, on average, mean harming patients. So much potential for conflict.
That's exactly what I said. Doctors don't understand AI and its place in medical practice, so they don't see how it could save them time for more valuable work. Instead they are afraid...
I'm all for machines replacing doctors, but I think it is important to remember that medical centers that make big bets on expensive machines frequently spin the truth to justify their purchases.
5 years ago, I was constantly being told about robot surgery and how it would make surgery safer by reducing mistakes. Hospitals that made these purchases made grand claims in an effort to attract patients. Unfortunately, the da vinci robot has continuously injured patients[1][2] while the marketing has made it so the machines continue to be installed in more hospitals as patients demand the robot surgeons.
Hospitals that have purchased a da vinci machine are very unlikely to come out and say it was a waste of money and is injuring patients, it makes them look like they are fools, and increases their liability. I really hope Watson is as advanced as described in the Wired article, but there have been plenty of popular science rags that similarly pimped robots as making surgery safer.
I think eventually robots will eventually make surgery safer, and watson-like computing will make better diagnoses, but there clearly was a mistake made in evaluating robots for surgery that I would rather not see made again.
So we might end up with better treatment, healthier lives, and lower preemiums. What, exactly, is the big scary downside? We are no longer subsidizing the luxurious high life of a particular educated elite?
Who do you think that the owners of said machines are?
More and more you will be owned by the rich, so prepare for living in a total dictatorship within your lifespan.
I think you misunderstand my point. Health care is an economic rent - a forced payment into a system that does nothing to advance society. It's like a thug who has seized the water supply and is demanding ever higher prices. What are you going to do? Not pay and die? The double-digit percentage of my salary that spend on fixing and maintaining the frail, degenerating body that evolution handed me I would rather spend on something else, if I could.
I would rather stay perfectly healthy, young forever, and not pay a cent. Wouldn't you? If moving towards that goal puts some doctors out of a job, well so be it. The end goal doesn't include their job anyway. Maybe they can go become engineers and create value, or go make better medical machines.
What obstacles are there in various nations to people opting to waive all medical liability in exchange for ultra-cheap access to Watson-based or -like diagnoses and robot-guided physical diagnostics and treatment? If I was indigent or a low socioeconomic participant in a Third World nation, then $0.10 USD visits to an autodoc that also dispenses the suggested curative treatment if I sign a waiver would be awfully attractive. Even a 30% success rate would be an astounding improvement against the status quo in many parts of the world, and many people would take it. And the autodoc software devs would get a constant stream of patient data to improve future versions upon. Not an especially pretty situation to contemplate (cue cries of exploitation of the desperate), but one we likely will have to face as these technologies continue to improve.
I'm interested in the fact that most of the discussion has centered around the possibility of Watson and other expert systems doing what is outlined in the article, rather than the societal ramifications mentioned (~70% "full" employment).
Perhaps I'm a luddite, but I wonder if this time it really "is different" or if we'll discover new ways to put human labor to work, just as we did when farmers left the land or when services overtook manufacturing as the primary sector creating jobs (in the developed world, at least).
Legal automation faces one giant hurdle. As any first year law student knows most of the time there is no "correct" answer. This isn't Jeopardy where the correct answer is "who is Winston Churchill". Law is a negotiation about whether or not the wife gets to keep the dog, whether or not the seller takes the risk of the shipment being held up in customs, or whether or not the purchase of A tech company should be subject to an earn out. None of these questions can (as yet) be readily answered by a formula. I very seriously doubt Watson can answer any of these questions.
Don't get me wrong - legal automation is coming but it is only replacing the least skilled. Paralegals, procurement officers and law students have the most to fear.
39 comments
[ 3.2 ms ] story [ 70.2 ms ] threadFull Disclosure: I just read a paper saying that only 10% of doctors are capable of bayesian reasoning, so I'm in a mood to pick on doctors today. I'll stop now.
This implementation of Watson is more than a query engine; it returns a confidence interval with its diagnoses and "knows" which questions to ask to improve the confidence level in those diagnoses.
1. Doctors on average miss 50% of questions needed to be asked on a medical interview.
2. In general , people are less likely to lie to a machine, when they would lie to their doctors if dealing with some socially sensitive issues.
This isn't after-the-singularity type science fiction where we're adding assumption upon assumption to several levels beyond what is possible.
This is applying a technology (that's already proven at performing similar complex deductive analysis over massive quantities of data) to a new discipline. Any assumptions about what we can do in the new discipline with the current technology are minimal and extremely likely overcomable.
Even if Watson isn't quite there, the way is pretty clear now. Linear improvements in the Watson approach will yield results.
Unlike with jeopardy, you do not have a huge well organized catalog of information lovingly curated by trivia worshiping nerds. It is a ridicules stumbling block but there you have it. Health records are not normalized and you can't touch them anyway, research is closer to human readable prose and filled with jargon - good luck developing NLP for it if you're "just a programmer". Hell, look at their textbooks and the way they study and are trained, that screams for computerized improvement and is still a tall order.
The way they are implemented is by people entering data in an orderly fashion, both those who built the systems ,and doctors . This somewhat limits the system, both from breadth of research and data entry time at the doctor.
But even with those limitations , those systems offer far better diagnoses , especially in the rare cases,than you average and even great doctor.
Watson is able to parse natural language and pull out possibly important information. It then applies some very state-of-the-art machine learning to derive the most likely contextual meaning of that information. That contextual meaning can be cross-referenced with billions of other pieces of weighted information to derive further insights.
I come from a family full of doctors. They tend to be smart people, but as Jeopardy showed, even the best player on earth is no match for a machine trained at that discipline.
Once we start adding in genetic information, that will allow machines like Watson to customize diagnoses and treatments based upon your specific genetic profile.
On the machine learning side, there is very little in the way of inference. Most of machine learning is an attempt to solve the statistical problem of predicting some value Y (e.g. a diagnosis) given a feature vector X.
To obtain data from medical texts (even with human assistance) and use this to form diagnoses of actual patients, would require solving both these problems. We would have to parse and represent the complex logical relationships that are expressed in academic papers. We would then have to have machine learning methods that can work with complex data (not just feature vectors) and combine inference with statistical learning.
And isn't doctors and insurance companies being impressed with results are another hint that watson has those capabilities(or is a very big fraud, something IBM wouldn't riskm) ?
For that matter it is hard to coordinate and record treatments if the process of diagnosis is too complicated or arcane for AI to understand.
It might, at the margins, make ‘miracle-worker’ doctors less so. Those doctors will be less able to demand exorbitant fees.
It will also make more-accurate cancer diagnosis available to more doctors, and therefore to more patients. But no one (in the medium term) will not use an oncologist because Watson exists.
These are all good things. In finance, the computers are doing most of the work but employment hasn’t waned (at least not because of the computers).
The underlying reason is that demand for improved results, in both fields, has no foreseeable upper bound. Demand grows to consume the new capacity. There is no upper limit on ‘better cancer diagnosis’, as there is no upper limit on ‘better financial returns’.
Put another way, Watson-assisted diagnosis should be thought of as a ‘new’ product, stoking new demand, in the same way better Macs drive demand every year.
Someday you’ll buy a cheap health robot at Walmart, but this development is not that.
(ps Tyler Cowen’s new book has some interesting stuff on this topic.)
Don't get me wrong, her doctor is a wonderful person that is very knowledgeable, but the way cancer medicine is done today is a lot of guess work and experimentation, with long lead times between measurements (blood tests and whatnot) that are hardly statistically valid and uber prone to tampering (used loosely in the quality control sense).
The existing workforce that will become far more productive are not the doctors. It's the nurses and physician's assistants. They provide value that is much harder to automate, although the Japanese certainly are trying to make progress in this area with humanoid robots to care for patients.
On top of the things you point out, cancer is all about the genes - the genes in the host as well as the myriad of mutations in the cancer.
Cancer treatment of the future will involve sequencing of the patient's genome and repeated sequencing of samples of the cancer itself. The computer will be needed to dynamically account for mutations and prescribe treatment for each one as it arises. Networks of medical computer systems will be able to consult at internet speeds on specific mutations to get suggestions for treatment that are built upon results of other computer prescriptions whose efficacy is much better recorded, processed, and correlated.
As you say, it will be the second tier of the medical industry that most benefits from this revolution. Any nurse who can follow the instructions given by the computer for gathering information, administering tests, feeding results back to the computer, and providing treatment could become a world class oncologist.
Probably doctors will still play a role in some level of oversight of what the computers are prescribing -- but that will fade as our trust level increases over time. Within ten years, computers will be assisting doctors on every diagnosis. Within twenty, they'll be assisting nurses, mostly replacing the need for doctor time. Within thirty years, you'll be able to perform your own interactions with a medical computer as easily as you Google today -- probably easier since it will interact with you using natural language.
You could also expand that idea to having it acting as a spam filter for medical publications. It could scan every old and new published paper for basic validity. This wouldn't prevent fraud, but it might raise the bar for publishing bad science.
Doctors might like reading research every once in a while. They don't like spending all their time on it. An overworked doctor should not care if he refers a patient to an overworked oncologist or asks watson first.
Computer vision applied in health care also automates tasks that are mind-numbing. Is it really necessary for highly-skilled doctors with decades of experience to look through a microscope?
Doctors also complain that they don't have enough time for their patients. Watson seems to be excellent at completing paperwork. Given that we hardly have enough doctors anywhere in the world, more AI will be good thing.
5 years ago, I was constantly being told about robot surgery and how it would make surgery safer by reducing mistakes. Hospitals that made these purchases made grand claims in an effort to attract patients. Unfortunately, the da vinci robot has continuously injured patients[1][2] while the marketing has made it so the machines continue to be installed in more hospitals as patients demand the robot surgeons.
Hospitals that have purchased a da vinci machine are very unlikely to come out and say it was a waste of money and is injuring patients, it makes them look like they are fools, and increases their liability. I really hope Watson is as advanced as described in the Wired article, but there have been plenty of popular science rags that similarly pimped robots as making surgery safer.
I think eventually robots will eventually make surgery safer, and watson-like computing will make better diagnoses, but there clearly was a mistake made in evaluating robots for surgery that I would rather not see made again.
[1] http://www.bloomberg.com/news/2013-10-08/robot-surgery-damag... [2] http://online.wsj.com/news/articles/SB1000142405270230470310...
I would rather stay perfectly healthy, young forever, and not pay a cent. Wouldn't you? If moving towards that goal puts some doctors out of a job, well so be it. The end goal doesn't include their job anyway. Maybe they can go become engineers and create value, or go make better medical machines.
My problem with machines is that sooner or later people cause changes while machines imprint status quo.
Perhaps I'm a luddite, but I wonder if this time it really "is different" or if we'll discover new ways to put human labor to work, just as we did when farmers left the land or when services overtook manufacturing as the primary sector creating jobs (in the developed world, at least).
Don't get me wrong - legal automation is coming but it is only replacing the least skilled. Paralegals, procurement officers and law students have the most to fear.