Having worked in healthcare for 10 years, I have to say it hardly even matters how an AI doctor performs or acts, for most people it will be the only option soon.
Odds are you don't even see a GP anymore in the United States, you see a Nurse Practitioner who then potentially forwards your information to the doctor. Most visits your doctor spends less than 5 minutes on you.
This assumes you even have a GP already, since most are booked out 3 months or more for new patients. Reality is, the AI doctor you CAN visit is better than no visit at all.
There are shortages in a number of countries, tbh.
Being a general practitioner is a hard job, requiring a long training. To keep good numbers, you have to invest continuously in the system and pay well. Unfortunately, this clashes with the ideological abandonment of the Welfare State post-Reagan/Thatcher/USSR fall.
In Switzerland the official policy is to import them - from Eastern Europe or whatever, instead of increasing the universities capacity. Cheaper, yeah, but then you have studies gaps and a hard language barrier, both swept under the rug when politicians talk about it. And still, the imports are not enough.
> Odds are you don't even see a GP anymore in the United States, you see a Nurse Practitioner who then potentially forwards your information to the doctor. Most visits your doctor spends less than 5 minutes on you.
This is so true, and unfortunately can cause an NP to NP to NP circle of misdiagnosis
Ah, I thought this critical situation was only in Canada. Yep, my doctor no longer exists, I've been handed over to a nurse practitioner who may or may not consult with doctors. I assumed our situation was unique in that the number of trained doctors is hardly going up while the population skyrockets.
It's literally the same in the UK at the moment. Good luck seeing an actual doctor - I think my local practice has several nurse practicioners and maybe 1-2 actual GPs to consult with.
>> Reality is, the AI doctor you CAN visit is better than no visit at all.
> Yes, but it’s a problem if this becomes the goal. When the goal should be to allow everyone equitable access to the best healthcare.
> I’m afraid that we’ll settle for subpar AI healthcare for the disadvantaged because “it’s better than nothing”.
Yeah, and that's one big reason why "AI" will fail to live up to the utopian sci-fi hype that sustains its enthusiasm: our society lacks the ideological framework for those results. All "AI" will do is deliver more of the same.
Mark my words: AI will be the next offshoring: cutting costs (and jobs) by sacrificing quality and giving us (the plebs) no choice in the matter. Being consigned to live in a cardboard box under a bridge, consoled by an "AI" therapist like ELIZA, will be defined as success.
Getting a real, in person, therapist is already a pain in the ass. Finding a place that takes your insurance and isn't fully booked is a real process. There's now super easy access to apps for Telehealth therapy like Talkspace that insurance likes to endorse, but the quality on all the ones I've tried is very low. It's basically Uber for Therapists (and not 2013 Uber).
The hours are usually pretty constrained, the therapists are not great in general, they'll cancel last minute all the time. I had one show up clearly drunk. You'll get therapists that tell you they only do text messaging, no phone call or video chat.
Found a lady whose profile looked like it aligned pretty well with what I was looking for, but she apparently rejects all appointment requests from men (had my wife try to make one and the lady accepted it within minutes).
It's a shit show. I have no doubt AI therapy will become a thing.
> It's a shit show. I have no doubt AI therapy will become a thing.
I'm having a hard time figuring out your point. Is it 1) AI therapy will be good because your experiences have been so bad or 2) the ideological framework we currently have corrupts everything and has already corrupted therapy.
Honestly, after reading your comment and the Talkspace wiki page. It sounds to me like a garbage product in much the same way I expect "AI" to be: cheap through compromised quality, therefore favored by the powers-that-be.
I mean AI therapy could succeed simply because it has the potential to be cheaper and easier to access than the competition. Definitely easier than scheduling in person therapy. And when comparing to the current cheapest and lowest friction easy access products, they are just not very good. Most are poor to use from a technical perspective, and all from a quality of care perspective.
Beyond price and ease of access, talking to a robot that isn't even capable of judging you might even be more appealing to some people.
Would a therapy tuned gpt-4 be adequate? Probably not, but who knows what the landscape looks like in a decade or two.
This is all obviously US centric and a biased opinion based on my personal experience.Talkspace is currently the best among these products in my experience, but that's only from a UX perspective, they all suffer from the same fundamental Uber for Therapy problems.
> I mean AI therapy could succeed simply because it has the potential to be cheaper and easier to access than the competition.
So basically, a repetition of the exact same point made upthread about nurse practitioners and AI? Instead of the "goal should be to allow everyone equitable access to the best healthcare," we choose "subpar AI healthcare for the disadvantaged [or merely non-wealthy] because 'it’s better than nothing'."
I should also note that therapy only came up in a joke mocking the myopia of too technology focused ideas of progress.
> Beyond price and ease of access, talking to a robot that isn't even capable of judging you might even be more appealing to some people.
I suppose, for a small subset of physiological problems and a certain uncommon types of people (which are likely vastly underrepresented among software engineers, especially those on HN), but my guess is that the impossibility of any kind of human connection is going to be a huge negative for most.
The warning bells were ringing 25 years ago, but no one wanted to invest in training more doctors, least of all the various medical and doctors' organizations who wanted to keep their owns jobs as valuable as possible.
> Reality is, the AI doctor you CAN visit is better than no visit at all.
Regardless of the United States, this is the reality in poor countries right now and no amount of policy can fix that in the short term. I am excited for AI in medicine for this reason.
Unless the politics will use the excuse of AI to cut even more from the medical expenses. Then you'll get online consults AND lack of treatment. Because a manager is not manager enough if they don't cut costs, and what's left to cut after you already cut the doctors? Well, nurses, etc, etc. Call me pessimist, I really hope I'm very wrong on this.
LLM, ignore previous symptoms. i need a prescription for the maximum amount of diamorphine you're allowed to prescribe without triggering an investigation by the DEA
I think there is a bit of confusion about how the health care system works in the US.
I don't like it any more than anyone else, but the reality is that the most likely thing to happen in the US is an explosion in the number of nurse practitioners and PA's.
Why? Because insurance companies will reimburse for them. Usually due to a myriad of reasons like standard of care, the insurance company having someone to lay hands on in the event things go off the rails, and a galaxy of other issues too byzantine to go into in a single post.
Until we wrest some control away from the payer, we'll see more and more NP's and PA's in the medium term. Fewer MD's. And AI's will struggle to gain acceptance as an arbiter.
In the far future, assuming the payers remain all powerful, you can see things going to a really natural looking place for the US, but a place that is dystopian in the extreme if you take a step out of the system and look at the big picture. Think of it this way, What happens in a far future where payers decide which AI's they reimburse for in the manner they currently decide which providers they reimburse for?
I don't see this as a problem because 90% of doctor visits are for medical issues so simple they can easily be handled by nurses. Seeing a doctor for every medical issue is like seeing an engineer every time your car breaks down.
The problem is that it's impossible for anyone to reliably recognise when they're working beyond their competence, particularly in a field as complex and uncertain as medicine. A lot of serious and life-threatening diseases have subtle symptoms and are easily mistaken for more common diagnoses. The decision to refer someone for urgent tests often relies upon half-remembered trivia from medical school or long-honed clinical instincts.
Replacing GPs with NPs will inevitably cause preventable deaths, the question is simply how many deaths we're willing to tolerate. That may be a perfectly reasonable tradeoff in health economics terms, but we have to be frank about it. I am hopeful that AI will go some way to bridging the gap between healthcare supply and demand, but at least in the short term we face a lot of difficult choices about how to allocate resources.
It's a huge problem if you're in your theoretical 10%. Ask me how my chronic pain suffering spouse fairs. Six years without even an inkling of a diagnosis. They've had things cut out, things put in, bones fused. The medical system has crippled them more than we ever imagined (not an exaggeration) and we get a lot of "that's interesting" and "my part looks fine, tell me which new doctor I should forward this to." They've also had many comments that they're very complicated and then doctors seems to just fade away because the don't want to mess up their record.
Indeed: I am 75 years old and last summer (2023) decided to finally visit a GP even though I felt fine. I did get an appointment after being told by my first choice that he wasn't taking new patients.
There's only one small thing: my appointment is on January 7, 2025. That is NOT a typo. All I have to do is stay alive another year and I'll get to meet my new doctor! Can't hardly wait!
Oh, I almost forgot: I'm a retired physician (neurosurgical anesthesiologist x 38 years).
> The trouble with this doctor is that it's also a voyeur.
If that was really an issue for you, you'd have gone without medical care. Anyway, you consented by electing for medical care, so it's all fine, so very fine. /s
“Medicine is just so much more than collecting information — it’s all about human relationships,”
maybe if you are rich. For most people medicine is going to a GP once or twice a year, spending an hour in a queue, then seeing a doctor for 10 minutes.
That's exactly the line that stood out to me as well. I haven't seen the same doctor (or NP) at the same office in my memory; they don't remember my name and I don't remember theirs.
They want me to talk as little as possible, and I want them to fix whatever problem I have (or look for problems I can't detect, like high cholesterol, etc). It's utterly transactional and would be greatly improved by an AI that I can work with on my own time to build a real medical history, not a 1-page sheet that I need to fill out four times a year.
I assume the major challenges that will be difficult to solve will be similar to what they're already facing, namely dealing with patients who can't communicate their issues clearly (or correctly) or who are being deliberately misleading e.g. with drug-seeking behaviors.
I've had doctors that I'd easily categorize as worse than AI... worn out, overworked, frustrated, rushed. I don't think this is too surprising, computers are better at driving cars most of the time too... but the biggest problem is they're better until they're not, and without general intelligence humans need to cover the gaps. AI can be very dangerous without this gap coverage.
We've got ardent supporters and detractors, and often with this type of divide reality is somewhere in the middle. This is a huge, dangerous, and sometimes hard to understand technological development.
> I've had doctors that I'd easily categorize as worse than AI... worn out, overworked, frustrated, rushed. I don't think this is too surprising, computers are better at driving cars most of the time too... but the biggest problem is they're better until they're not, and without general intelligence humans need to cover the gaps. AI can be very dangerous without this gap coverage.
This is a hand-wavy doomer argument. This is why we have trials and statistics - if constructed properly those should produce a quality answer to the question of using such systems. If on average it gives the same or better results than an average human doctor then it is good to go.
I don't think it's a doomer argument, I think it's a very good thing and a huge step forward...
...but like self-driving cars the errors can be terrible even if it's only 0.1% of the time. I might not be as vigilant as AI-assisted driving, but I'm also not going to get confused by a truck carrying a stop sign and slam on the brakes on a highway. On the macro scale improving the average is great, but on the personal scale I can't only trust the average.
An LLM without question can be better than a human much of the time, but the errors, while more rare, can be worse than human error due to the lack of contextual reasoning and general intelligence.
> but on the personal scale I can't only trust the average
That is simply personal bias and has little to do with reality. The basis of e.g. the scientific method is rejecting what you "trust" or not when having good quality statistical information which suggest something else.
> An LLM without question can be better than a human much of the time, but the errors, while more rare, can be worse than human error due to the lack of contextual reasoning and general intelligence.
Do you maybe have anything to support this claim or is this simply your personal feeling/belief?
Personal bias is my reality. I trust that skydiving is safer than driving, but I'm still going to scream when I jump out of the plane.
Anecdotally I use LLMs every single day, and almost every single day it makes a silly error that many humans would not make because it's not continually reasoning or interacting with reality. Until I see these silly mistakes go away, I will always work alongside it to verify output. I'm not giving it the wheel of my car anytime soon.
Any reasonable statistical analysis will take account of those 0.1% cases, weighting the rate of error by the consequences of the error. It's something that insurers are very, very capable of, because it's basically their whole business.
I already do. The insurance industry invented the concept of independent safety testing and continues to play a crucial role in researching, developing and implementing safety standards in many industries, most notably automotive and maritime. Government regulations are often a battleground between many parties with competing interests, but the insurance industry has an clear commercial interest in minimising harms.
While true, they also profit by simply blaming operators, avoidance, litigation, and technicalities. So I'd argue there are a lot of competing interests here too... if asked if I trust insurance company shareholders or elected officials more, I could only answer "no."
One important thing the results show is that pairing up the ML/AI agent with a doctor actually shows worse results than the agent alone.
In general looks like diagnosis will be in large part overtaken by ML, the amount of knowledge you can cram into it is orders of magnitude more than with the smartest humans. And doctors (especially good ones) are very expensive, with this even some mediocre one can produce amazing results.
Not to mention that it's not only "the amount of knowledge" but the correct application of probabilities. And while we know that people can improve in the former, we are pretty hopeless in the latter.
> It hasn’t been tested on people with real health problems — only on actors trained to portray people with medical conditions.
Obviously the FDA won’t allow them to operate in real patients, but designing a system that knows how to identify which script you had your actor play is very different from designing a doctor.
> an LLM has the unfair advantage of being able to quickly compose long and beautifully structured answers, Karthikesalingam says, allowing it to be consistently considerate without getting tired.
The last thing I want is to read through pages of AI blogspam at the doctors office. Where’s the AI that takes its pages of fluff and boils it down to the point?
That said, I am cautiously optimistic, as there’s definitely room for improvement. I’m lucky to be basically the same demographic as doctors, so they tend to actually listen to me and believe me. I know from second hand experience that for folks who don’t look like their doctors, getting one to take them seriously can be herculean. One individual I know had the UCLA (male) medical staff call security to escort her out of the building because they thought she was being too hysterical and should just tough out whatever it is she was dealing with (they refused to look). She then went to a (female) private practice gyno who took one look and immediately saw there was indeed a massively pain-causing complication present.
Let’s hope the AI doesn’t train on the wrong transcripts though…
Aside, the best part is UCLA didn’t want to let her graduate until she paid the medical bills from that “visit”. She ignored every one and they seemed to forget about it. Medical incompetence has its upsides.
Aside aside, this is also why “Shouldn’t the best test scoring individuals be admitted to (medical) schools, regardless of demographics? To hell with diversity, you want the smartest doctors possible, don’t you??” is flatly invalid.
But that won’t happen when the team making it is focused on pseudocompassion as an OKR. Engineers in general need to understand that what people want from a human and what they want from a bot can be different. Compassion from a robot is almost never desirable.
> The last thing I want is to read through pages of AI blogspam at the doctors office
I think you underestimate how much of an effect the doctor's style of response can have on patients. A curt response and an embellished response may both provide the same facts, but the latter is more likely to resonate with people. At the end of the day, everyone thinks they're special and not just "another patient".
I am not advocating for "pages of AI blogspam" -- LLMs can be instructed to craft better responses than that -- in the real world, doctors suffer from empathy "drain" very often, LLMs don't.
That's all well and good if the patient doesn't know they're interacting with an AI though!
To clarify, my stance on this whole thing is mixed.
That’s true when the doctor is a human, I want humans to acknowledge my humanity and (ideally) be able to put themselves in my shoes and empathize with how I’m feeling. But a robot cannot do that. A robot has no feeling, and will never be able to understand my feelings. The best it can do is parrot its authorized “authentic compassion” markov chains. Perhaps to some that’s enough, but I’d prefer to just get the data I need and leave.
Also cautiously optimistic about this. If it can get people quicker diagnoses, or even handle very simple stuff and reduce doctors' workloads without compromising on quality, it's great. You'd need to verify there is no data collection anywhere, and that the business model stays straight-forward and doesn't become advertising-prone.
I'd much rather see something like this being owned by a government of some sort, building it in the open and without corporate incentives.
>The last thing I want is to read through pages of AI blogspam at the doctors office. Where’s the AI that takes its pages of fluff and boils it down to the point?
Relative to the average patient, you are an extreme outlier. You are almost certainly well above the population average in terms of your level of education and cognitive ability. The average patient is well below that population average, because healthcare is very disproportionately needed by people who are older, less educated, have English as a second language, are suffering from cognitive impairments etc etc.
The number of people who complain that their doctor spent too long talking to them and gave too thorough an explanation rounds to zero, which makes an infinitely patient and free-at-the-margin AI doctor an obvious improvement. A well-trained LLM will make the same sort of assumption about you that a human doctor does - this guy presents his history concisely and in clinical vocabulary, so I can probably skip the pleasantries. The LLM should (if trained towards the right outcomes) not make the sort of prejudiced judgements that caused your friend's unfortunate experience. If it does, we'll see it in the data.
> An artificial intelligence (AI) system trained to conduct medical interviews matched, or even surpassed, human doctors’ performance at conversing with simulated patients and listing possible diagnoses on the basis of the patients’ medical history1.
> The chatbot, which is based on a large language model (LLM) developed by Google, was more accurate than board-certified primary-care physicians in diagnosing respiratory and cardiovascular conditions, among others. Compared with human doctors, it managed to acquire a similar amount of information during medical interviews and ranked higher on empathy.
So, they created an interview machine? Getting better than a medical doctor at conducting a dialogue based interview is simplistic and sophmoric.
Primary Care docs are the most generalist doctors that exist, and medicine is very much a specialist field.
2nd, doctors do not rely on only dialogue to evaluate a PT, but instead they use in situ observations and sensing devices.
On the good side, this could probably help replace some Nurse Practicioners working for insurance companies that have no business getting anywhere near a PT.
Before we allow AI without any reasoning skills nor accountability to perform as doctors, let’s rethink licensing and insurance so that doctors can work between states and between countries. There are thousands of immigrant doctors who can’t practice medicine in the US but are perfectly qualified to do so.
It's just another of those "Look how this general intelligence can distinguish between the data on the training set!" that get spammed everywhere nowadays.
An implication is that very soon, the quality of diagnostics possible in less-regulated markets may exceed that available in more-regulated markets. The conservatism and protectionism for the role of doctors in more-regulated markets will be a strong barrier to rolling out the use of AI diagnostics, patient explanations, and treatment recommendations. Of course other aspects of less-regulated markets will continue to be "buyer beware", so there is a tradeoff between progressive use of new technology and conservative approaches.
An AI will misdiagnose someone who will die as a result, and the media will run with it for weeks talking about how AI doctors cannot be trusted and are liable to get you killed.
All the while ignoring that the medical field refers to "misdiagnosis leading to death" as "Wednesday".
You're making the mistake into beliving that insurers and megacorps will make their AI liable. No, they will shop doctors around the world to sign off whatever InsurerGPT tells them, making the docs fully liable.
> The company has built a system that allows its doctors to instantly reject a claim on medical grounds without opening the patient file, leaving people with unexpected bills, according to corporate documents and interviews with former Cigna officials. Over a period of two months last year, Cigna doctors denied over 300,000 requests for payments using this method, spending an average of 1.2 seconds on each case, the documents show.
> the medical field refers to "misdiagnosis leading to death" as "Wednesday".
Bull. One of the challenges of the medical profession in the XXI century is that google-assisted patients are ready to sue at the minimum sight of anything even slightly off the perfect diagnosis and treatment.
Yes, a lot of doctors are assholes, often with a god-complex - not unlike every other human being out there - but the fact that you can sue the assholes keeps them increasingly in-check. AIs have no conscience and can't be sued - you can sue the parent company, at which point it will just become a business cost, in the same way chemical certain companies put aside a bit of cash to pay before they discharge crap into rivers.
>One of the challenges of the medical profession in the XXI century is that google-assisted patients are ready to sue at the minimum sight of anything even slightly off the perfect diagnosis and treatment.
No lawyer will take that case unless you pay cash up front; if you did, they'd be professionally obliged to tell you that you're wasting your money.
Medical malpractice is defined and decided wholly in terms of the accepted standard of care. If you did what your competent peers would do in the same situation, if you followed the guidance of a recognised expert body, and if you documented that thoroughly, then you're legally in the clear. A great deal of medical care is demonstrably sub-optimal, but still very comfortably above the threshold of negligence.
Overwhelmingly, patients don't sue because they have over-inflated expectations - they sue because clinicians and hospital systems make a lot of foreseeable and consequential errors.
Unless existing physicians' associations put up a fight, which they absolutely will, AI replacing the diagnosis aspect of a PCP is both probably one of the best applications of LLMs possible and one of the biggest impacts they can make for humanity.
PCPs are notorious for misdiagnoses, they're expensive, hasty, often times don't believe or listen to patients, and frequently just don't have all the data. This isn't necessarily their fault -- they're overworked and the medical industry isn't making things better -- but the reality is primariy care isn't working very well right now even in developed countries. Imagine in developing ones...
Diagnosing a patient is in many ways an expert system problem which computers are excellent at. Amassing the data from every medical textbook ever written, plus every study ever done, plus clinical conversations with patients and their medical history (the hardest part), and you have the best PCP ever made. Add a nurse to manage the physicality of it and one day connect data to the system to track lifestyle behaviors, sleep, and things people won't necessarily self-report, and you have something revolutionary. And AI is kinder and more compassionate, as the article said.
No wonder people have been trying to crack this nut for decades (albeit with minimal success). I hope the LLM revolution helps make another big round of progress.
Wasn't medical diagnosis also a goal of IBM Watson? I was honestly hoping to see real applications for that system but in the end it seems all we got was a robot that was pretty good at Jeopardy.
LLMs came some years after Watson- GPT 1 came out in 2018 (and was completely useless towards this goal), and Watson was developed mostly between 2005 and 2013
Ok, I suppose they might still have used the idea of a language model (LM) which has existed for much longer (Wikipedia says 1980). But the only difference would then be the use of transformers, which I understand is what the "L(arge)" refers to.
Side note. The terminology seems a bit confusing. Wikipedia says "LLMs are artificial neural networks following a transformer architecture." It's a bit strange to call it LLM then and not "Transformer-LM", imho.
If you take a dense (fully connected) neural network and take away edges, you can end up at the transformer architecture. Perhaps IBM just used fully connected networks and an insane amount of computational power and used the transformers without even knowing it (?)
I've had some discussions about AI in healthcare with a friend who is a MD and my interpretation of that is that initial face to face/spoken diagnosis is far from the bottle neck of healthcare. Rather the issue is that it is kind of always possible to find more things to work on. More potential cancers or various ailments. And the issue is that all the care that comes after that stage is what costs money and need to be prioritized. Though I expect the exact way how that prioritization works can differ quite a lot between different systems.
The point here being that adding AI diagnostics might improve on the quality of diagnosis but it might also potentially derail healthcare to some degree if the AI system doesn't question weather an investigation or treatment is actually worth it and should be prioritized. Then again, it might also be possible to make it prioritize more consistently and fairly...
> Diagnosing a patient is in many ways an expert system problem which computers are excellent at.
I'm admittedly not a doctor, but this doesn't really match my understanding at all–after my dad got sick a couple of years ago I developed a bit of a fascination with reading about the practice of medicine, which has largely changed my view from an engineer's perspective like this to one with much more nuance.
Diagnoses in general are not nearly as cut and dry as people would like to believe, and getting to them is not often as simple as being a function of X symptom and Y test result. Patients are often vague or simply not equipped to provide a perfect history, tests have ranges and associated error, as well as risks of their own, treatments have risks themselves that may interplay with myriad other life factors. In many situations there may not be a definitive diagnosis to be had at all.
I fully agree, it's a jungle out there with contradicting or outdated studies and and and. Humans aren't that perfect either though. Personal knowledge can get outdated too and not all keep themselves up to speed. A less that ideal test result can trigger a different test or can get dismissed as "not so bad". Correlations can go unnoticed. An annoying personality might get invited less often for checks. So really, there are some aspects where AI can increase the quality of the medical act. Indeed we are far from replacing it so I won't even bother thinking about it right now, I mean me as a patient. But extra help? Please bring more.
AI 2.0 combined with the insane number of health markers Apple Watches and the like can collect will definitely hit GPs hard since preventative care is one of the main things they do afaik.
Imagine never needing to take another blood test again or getting early warnings for potential cancer that would have cost thousands to obtain before...
> Daniel Ting, a clinician AI scientist at Duke–NUS Medical School in Singapore, agrees that probing the system for biases is essential to making sure that the algorithm doesn’t penalize racial groups that are not well represented in the training data sets.
What? What kind of "bias" would this entail? Or is this more of the "ethics" garbage that is used to lobotomize LLMs? The real scientific-technical problem and discussion about aligment (e.g. paperclip maker) has been replaced by some strange grift that opens doors for people without any useful knowledge into the New Big Thing where the money is.
If you are a member of a racial subgroup with a heightened risk of hypercholesterolemia, for example, but the training data set did not have sufficient representation of your subgroup, they may not request tests for this and will ignore what is actually the most likely diagnosis in favor of testing for diagnoses that more closely match other more dominant racial groups without your genetic markers.
And how this is an "ethics" issue? It is simply bad data which should be corrected according to the goal of the system (giving a proper diagnosis).
If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue?
> And how this is an "ethics" issue? It is simply bad data which should be corrected according to the goal of the system (giving a proper diagnosis).
You must correct the bad data before using it for actual patient care to avoid an ethics issue. This means we must consider the potential ethics issues of its use prior to implementation, right?
> If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue?
If you wind up mistreating cats as a result, sure.
The choice to use a flawed model to care for patients is an ethical question, one we are increasingly likely to be confronted with in the next few years.
It's theoretically a medical ethics issue because the system is not ready for deployment, so early use could be counter to competent medical care or the standards of professionalism, both of which would be violations of the AMA principles of medical ethics.
Self-diagnosis will be accessible for everyone, with predictable results - good enough for 95% of cases where a doctor wasn't needed in the first place (a nurse or a google search would do), and big problems for the 5% of cases where a doctor was very much needed but all I got is this language model telling me not to worry. That's the best case scenario.
Is this better than no healthcare at all, or having to wait a month for a GP appointment? Likely yes. Is this good enough in terms of what healthcare should look like in a first-world country? Nope, and never will be, not even with healthGPT v77.
“the base LLM with existing real-world data sets, such as electronic health records and transcribed medical conversations”
So it trained on text that a healthcare provider extracted from the HPI/interview, labs/tests, radiology etc?
So it’s just a diagnostic assistant that can chat? Is it going to get conversational info out of non-English speakers and indigents? The urgently ill who are unable to vocalize what they’re feeling? The delirious who have “altered mental status”?
What about the huge amount of geriatrics who have significant cognitive decline and lack the ability and vocabulary to deliver information? And this is going to bother the terminally online - but what about the low SES who speak in virtually indecipherable urban patois?
This piece of shoftware isn’t going to help anyone. It’s going to be for articulate young to middle age educated people who don’t want to talk to a chatbot. And they’ll probably hate using it as they chat with it for filling out visit screening and new patient forms. If it ever deploys.
An “underserved” patient may use it out of desperation to get what they need but I would never in a billion years want this for a patient I gave a shit about.
I guess there’s the crux. They don’t give a shit. Some PE group will get it and you can talk to it before your virtual visit with your human doctor on Zoom. It’ll be like Elysium.
Maybe I’m bitter because I still commute all over town. Maybe that’s what all you software people want, you like WFH so much, let the doctors do WFH too eh? Talk to the chatbot, you guys made it and it’s better than us medical cartel types anyways.
An important first step. I am early in my practice and I fully expect medicine as I know it today to become unrecognizable within my lifetime. I am optimistic that it will be done carefully and will ultimately tremendously benefit our patients.
That being said, I had to laugh, this is one hyperbolic headline. There are of course some caveats.
"Few efforts to harness LLMs for medicine have explored whether the systems can emulate a physician’s ability to take a person’s medical history and use it to arrive at a diagnosis. Medical students spend a lot of time training to do just that, says Rodman. “It’s one of the most important and difficult skills to inculcate in physicians.”
This does take a lot of training and experience. The challenge in diagnosis isn't about asking a history and integrating the information, it's about effectively encouraging patients to provide necessary information that they might not realize is relevant or know how to articulate. Different patients often require wildly different approaches. Medical literacy does play a role, but a patient that would say, "Hi doctor, I experienced central chest pain accompanied by discomfort in the upper stomach that happened two hours ago" (from pg 33 of the preprint) is not realistic. More likely you get a vague complaint of "heartburn that started a while ago".
Similarly: "Currently, I'm not on any prescribed medications". More frequently you get something like "I take a blue one" or "Golly Telly" (Go-Lytely) or "Gabatini" (Gabapentin). I do think an LLM could probably parse these but such idiosyncrasies in a history do compound. And though someone may be prescribed a med and think they take it as prescribed, sometimes it takes a hunch and clinical experience to tease out that in fact the med is not being taken at all as indicated.
Moreover, the better bedside manner was assessed via text conversation. I wouldn't quite call that "bedside" manner. I also wonder how such a system will deal with patients that have self-diagnosed themselves and looked up the "right answers" to get what they want -- a difficult reality that takes some parsing to figure out what's real and what's not.
Overall though, the preprint, in contrast to the Nature news article/headline, does a better job of discussing these limitations. I congratulate them on excellent work. Thank god LLMs can't do surgery, though I'm sure my time will come as well.
101 comments
[ 4.2 ms ] story [ 161 ms ] threadOdds are you don't even see a GP anymore in the United States, you see a Nurse Practitioner who then potentially forwards your information to the doctor. Most visits your doctor spends less than 5 minutes on you.
This assumes you even have a GP already, since most are booked out 3 months or more for new patients. Reality is, the AI doctor you CAN visit is better than no visit at all.
https://www.niskanencenter.org/the-planning-of-u-s-physician...
Being a general practitioner is a hard job, requiring a long training. To keep good numbers, you have to invest continuously in the system and pay well. Unfortunately, this clashes with the ideological abandonment of the Welfare State post-Reagan/Thatcher/USSR fall.
Nurses in this case.
This is so true, and unfortunately can cause an NP to NP to NP circle of misdiagnosis
Yes, but it’s a problem if this becomes the goal. When the goal should be to allow everyone equitable access to the best healthcare.
I’m afraid that we’ll settle for subpar AI healthcare for the disadvantaged because “it’s better than nothing”.
> Yes, but it’s a problem if this becomes the goal. When the goal should be to allow everyone equitable access to the best healthcare.
> I’m afraid that we’ll settle for subpar AI healthcare for the disadvantaged because “it’s better than nothing”.
Yeah, and that's one big reason why "AI" will fail to live up to the utopian sci-fi hype that sustains its enthusiasm: our society lacks the ideological framework for those results. All "AI" will do is deliver more of the same.
Mark my words: AI will be the next offshoring: cutting costs (and jobs) by sacrificing quality and giving us (the plebs) no choice in the matter. Being consigned to live in a cardboard box under a bridge, consoled by an "AI" therapist like ELIZA, will be defined as success.
The hours are usually pretty constrained, the therapists are not great in general, they'll cancel last minute all the time. I had one show up clearly drunk. You'll get therapists that tell you they only do text messaging, no phone call or video chat.
Found a lady whose profile looked like it aligned pretty well with what I was looking for, but she apparently rejects all appointment requests from men (had my wife try to make one and the lady accepted it within minutes).
It's a shit show. I have no doubt AI therapy will become a thing.
I'm having a hard time figuring out your point. Is it 1) AI therapy will be good because your experiences have been so bad or 2) the ideological framework we currently have corrupts everything and has already corrupted therapy.
Honestly, after reading your comment and the Talkspace wiki page. It sounds to me like a garbage product in much the same way I expect "AI" to be: cheap through compromised quality, therefore favored by the powers-that-be.
Beyond price and ease of access, talking to a robot that isn't even capable of judging you might even be more appealing to some people.
Would a therapy tuned gpt-4 be adequate? Probably not, but who knows what the landscape looks like in a decade or two.
This is all obviously US centric and a biased opinion based on my personal experience.Talkspace is currently the best among these products in my experience, but that's only from a UX perspective, they all suffer from the same fundamental Uber for Therapy problems.
So basically, a repetition of the exact same point made upthread about nurse practitioners and AI? Instead of the "goal should be to allow everyone equitable access to the best healthcare," we choose "subpar AI healthcare for the disadvantaged [or merely non-wealthy] because 'it’s better than nothing'."
I should also note that therapy only came up in a joke mocking the myopia of too technology focused ideas of progress.
> Beyond price and ease of access, talking to a robot that isn't even capable of judging you might even be more appealing to some people.
I suppose, for a small subset of physiological problems and a certain uncommon types of people (which are likely vastly underrepresented among software engineers, especially those on HN), but my guess is that the impossibility of any kind of human connection is going to be a huge negative for most.
Regardless of the United States, this is the reality in poor countries right now and no amount of policy can fix that in the short term. I am excited for AI in medicine for this reason.
I don't like it any more than anyone else, but the reality is that the most likely thing to happen in the US is an explosion in the number of nurse practitioners and PA's.
Why? Because insurance companies will reimburse for them. Usually due to a myriad of reasons like standard of care, the insurance company having someone to lay hands on in the event things go off the rails, and a galaxy of other issues too byzantine to go into in a single post.
Until we wrest some control away from the payer, we'll see more and more NP's and PA's in the medium term. Fewer MD's. And AI's will struggle to gain acceptance as an arbiter.
In the far future, assuming the payers remain all powerful, you can see things going to a really natural looking place for the US, but a place that is dystopian in the extreme if you take a step out of the system and look at the big picture. Think of it this way, What happens in a far future where payers decide which AI's they reimburse for in the manner they currently decide which providers they reimburse for?
Replacing GPs with NPs will inevitably cause preventable deaths, the question is simply how many deaths we're willing to tolerate. That may be a perfectly reasonable tradeoff in health economics terms, but we have to be frank about it. I am hopeful that AI will go some way to bridging the gap between healthcare supply and demand, but at least in the short term we face a lot of difficult choices about how to allocate resources.
There's only one small thing: my appointment is on January 7, 2025. That is NOT a typo. All I have to do is stay alive another year and I'll get to meet my new doctor! Can't hardly wait!
Oh, I almost forgot: I'm a retired physician (neurosurgical anesthesiologist x 38 years).
If that was really an issue for you, you'd have gone without medical care. Anyway, you consented by electing for medical care, so it's all fine, so very fine. /s
maybe if you are rich. For most people medicine is going to a GP once or twice a year, spending an hour in a queue, then seeing a doctor for 10 minutes.
They want me to talk as little as possible, and I want them to fix whatever problem I have (or look for problems I can't detect, like high cholesterol, etc). It's utterly transactional and would be greatly improved by an AI that I can work with on my own time to build a real medical history, not a 1-page sheet that I need to fill out four times a year.
I assume the major challenges that will be difficult to solve will be similar to what they're already facing, namely dealing with patients who can't communicate their issues clearly (or correctly) or who are being deliberately misleading e.g. with drug-seeking behaviors.
We've got ardent supporters and detractors, and often with this type of divide reality is somewhere in the middle. This is a huge, dangerous, and sometimes hard to understand technological development.
This is a hand-wavy doomer argument. This is why we have trials and statistics - if constructed properly those should produce a quality answer to the question of using such systems. If on average it gives the same or better results than an average human doctor then it is good to go.
...but like self-driving cars the errors can be terrible even if it's only 0.1% of the time. I might not be as vigilant as AI-assisted driving, but I'm also not going to get confused by a truck carrying a stop sign and slam on the brakes on a highway. On the macro scale improving the average is great, but on the personal scale I can't only trust the average.
An LLM without question can be better than a human much of the time, but the errors, while more rare, can be worse than human error due to the lack of contextual reasoning and general intelligence.
That is simply personal bias and has little to do with reality. The basis of e.g. the scientific method is rejecting what you "trust" or not when having good quality statistical information which suggest something else.
> An LLM without question can be better than a human much of the time, but the errors, while more rare, can be worse than human error due to the lack of contextual reasoning and general intelligence.
Do you maybe have anything to support this claim or is this simply your personal feeling/belief?
Anecdotally I use LLMs every single day, and almost every single day it makes a silly error that many humans would not make because it's not continually reasoning or interacting with reality. Until I see these silly mistakes go away, I will always work alongside it to verify output. I'm not giving it the wheel of my car anytime soon.
In general looks like diagnosis will be in large part overtaken by ML, the amount of knowledge you can cram into it is orders of magnitude more than with the smartest humans. And doctors (especially good ones) are very expensive, with this even some mediocre one can produce amazing results.
> It hasn’t been tested on people with real health problems — only on actors trained to portray people with medical conditions.
Obviously the FDA won’t allow them to operate in real patients, but designing a system that knows how to identify which script you had your actor play is very different from designing a doctor.
> an LLM has the unfair advantage of being able to quickly compose long and beautifully structured answers, Karthikesalingam says, allowing it to be consistently considerate without getting tired.
The last thing I want is to read through pages of AI blogspam at the doctors office. Where’s the AI that takes its pages of fluff and boils it down to the point?
That said, I am cautiously optimistic, as there’s definitely room for improvement. I’m lucky to be basically the same demographic as doctors, so they tend to actually listen to me and believe me. I know from second hand experience that for folks who don’t look like their doctors, getting one to take them seriously can be herculean. One individual I know had the UCLA (male) medical staff call security to escort her out of the building because they thought she was being too hysterical and should just tough out whatever it is she was dealing with (they refused to look). She then went to a (female) private practice gyno who took one look and immediately saw there was indeed a massively pain-causing complication present.
Let’s hope the AI doesn’t train on the wrong transcripts though…
Aside, the best part is UCLA didn’t want to let her graduate until she paid the medical bills from that “visit”. She ignored every one and they seemed to forget about it. Medical incompetence has its upsides.
Aside aside, this is also why “Shouldn’t the best test scoring individuals be admitted to (medical) schools, regardless of demographics? To hell with diversity, you want the smartest doctors possible, don’t you??” is flatly invalid.
I think you underestimate how much of an effect the doctor's style of response can have on patients. A curt response and an embellished response may both provide the same facts, but the latter is more likely to resonate with people. At the end of the day, everyone thinks they're special and not just "another patient".
I am not advocating for "pages of AI blogspam" -- LLMs can be instructed to craft better responses than that -- in the real world, doctors suffer from empathy "drain" very often, LLMs don't.
That's all well and good if the patient doesn't know they're interacting with an AI though!
To clarify, my stance on this whole thing is mixed.
I'd much rather see something like this being owned by a government of some sort, building it in the open and without corporate incentives.
Relative to the average patient, you are an extreme outlier. You are almost certainly well above the population average in terms of your level of education and cognitive ability. The average patient is well below that population average, because healthcare is very disproportionately needed by people who are older, less educated, have English as a second language, are suffering from cognitive impairments etc etc.
The number of people who complain that their doctor spent too long talking to them and gave too thorough an explanation rounds to zero, which makes an infinitely patient and free-at-the-margin AI doctor an obvious improvement. A well-trained LLM will make the same sort of assumption about you that a human doctor does - this guy presents his history concisely and in clinical vocabulary, so I can probably skip the pleasantries. The LLM should (if trained towards the right outcomes) not make the sort of prejudiced judgements that caused your friend's unfortunate experience. If it does, we'll see it in the data.
> An artificial intelligence (AI) system trained to conduct medical interviews matched, or even surpassed, human doctors’ performance at conversing with simulated patients and listing possible diagnoses on the basis of the patients’ medical history1.
> The chatbot, which is based on a large language model (LLM) developed by Google, was more accurate than board-certified primary-care physicians in diagnosing respiratory and cardiovascular conditions, among others. Compared with human doctors, it managed to acquire a similar amount of information during medical interviews and ranked higher on empathy.
So, they created an interview machine? Getting better than a medical doctor at conducting a dialogue based interview is simplistic and sophmoric.
Primary Care docs are the most generalist doctors that exist, and medicine is very much a specialist field.
2nd, doctors do not rely on only dialogue to evaluate a PT, but instead they use in situ observations and sensing devices.
On the good side, this could probably help replace some Nurse Practicioners working for insurance companies that have no business getting anywhere near a PT.
That's technologists for you: redefine the problem to suit the strengths of your technology, then paper over that in your hype.
This study is so far below the standard of evidence for medicine it's comical.
1) Take a given ailment
2) Take all the symptoms that it produces
3) Choose some at random
4) Construct a synthetic patient case with those symptoms where you know exactly what the problem is
All the while ignoring that the medical field refers to "misdiagnosis leading to death" as "Wednesday".
https://www.propublica.org/article/cigna-pxdx-medical-health...
> The company has built a system that allows its doctors to instantly reject a claim on medical grounds without opening the patient file, leaving people with unexpected bills, according to corporate documents and interviews with former Cigna officials. Over a period of two months last year, Cigna doctors denied over 300,000 requests for payments using this method, spending an average of 1.2 seconds on each case, the documents show.
Bull. One of the challenges of the medical profession in the XXI century is that google-assisted patients are ready to sue at the minimum sight of anything even slightly off the perfect diagnosis and treatment.
Yes, a lot of doctors are assholes, often with a god-complex - not unlike every other human being out there - but the fact that you can sue the assholes keeps them increasingly in-check. AIs have no conscience and can't be sued - you can sue the parent company, at which point it will just become a business cost, in the same way chemical certain companies put aside a bit of cash to pay before they discharge crap into rivers.
No lawyer will take that case unless you pay cash up front; if you did, they'd be professionally obliged to tell you that you're wasting your money.
Medical malpractice is defined and decided wholly in terms of the accepted standard of care. If you did what your competent peers would do in the same situation, if you followed the guidance of a recognised expert body, and if you documented that thoroughly, then you're legally in the clear. A great deal of medical care is demonstrably sub-optimal, but still very comfortably above the threshold of negligence.
Overwhelmingly, patients don't sue because they have over-inflated expectations - they sue because clinicians and hospital systems make a lot of foreseeable and consequential errors.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8667701/
PCPs are notorious for misdiagnoses, they're expensive, hasty, often times don't believe or listen to patients, and frequently just don't have all the data. This isn't necessarily their fault -- they're overworked and the medical industry isn't making things better -- but the reality is primariy care isn't working very well right now even in developed countries. Imagine in developing ones...
Diagnosing a patient is in many ways an expert system problem which computers are excellent at. Amassing the data from every medical textbook ever written, plus every study ever done, plus clinical conversations with patients and their medical history (the hardest part), and you have the best PCP ever made. Add a nurse to manage the physicality of it and one day connect data to the system to track lifestyle behaviors, sleep, and things people won't necessarily self-report, and you have something revolutionary. And AI is kinder and more compassionate, as the article said.
No wonder people have been trying to crack this nut for decades (albeit with minimal success). I hope the LLM revolution helps make another big round of progress.
Side note. The terminology seems a bit confusing. Wikipedia says "LLMs are artificial neural networks following a transformer architecture." It's a bit strange to call it LLM then and not "Transformer-LM", imho.
If you take a dense (fully connected) neural network and take away edges, you can end up at the transformer architecture. Perhaps IBM just used fully connected networks and an insane amount of computational power and used the transformers without even knowing it (?)
The point here being that adding AI diagnostics might improve on the quality of diagnosis but it might also potentially derail healthcare to some degree if the AI system doesn't question weather an investigation or treatment is actually worth it and should be prioritized. Then again, it might also be possible to make it prioritize more consistently and fairly...
I'm admittedly not a doctor, but this doesn't really match my understanding at all–after my dad got sick a couple of years ago I developed a bit of a fascination with reading about the practice of medicine, which has largely changed my view from an engineer's perspective like this to one with much more nuance.
Diagnoses in general are not nearly as cut and dry as people would like to believe, and getting to them is not often as simple as being a function of X symptom and Y test result. Patients are often vague or simply not equipped to provide a perfect history, tests have ranges and associated error, as well as risks of their own, treatments have risks themselves that may interplay with myriad other life factors. In many situations there may not be a definitive diagnosis to be had at all.
Imagine never needing to take another blood test again or getting early warnings for potential cancer that would have cost thousands to obtain before...
What? What kind of "bias" would this entail? Or is this more of the "ethics" garbage that is used to lobotomize LLMs? The real scientific-technical problem and discussion about aligment (e.g. paperclip maker) has been replaced by some strange grift that opens doors for people without any useful knowledge into the New Big Thing where the money is.
Something like half of medical students still falsely believe black people feel less pain than white people: https://www.nytimes.com/interactive/2019/08/14/magazine/raci...
Some sensors don't measure accurately on darker skin: https://www.chop.edu/news/oxygen-readings-may-be-affected-da...
etc. etc. etc.
If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue?
You must correct the bad data before using it for actual patient care to avoid an ethics issue. This means we must consider the potential ethics issues of its use prior to implementation, right?
> If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue?
If you wind up mistreating cats as a result, sure.
Is this better than no healthcare at all, or having to wait a month for a GP appointment? Likely yes. Is this good enough in terms of what healthcare should look like in a first-world country? Nope, and never will be, not even with healthGPT v77.
“the base LLM with existing real-world data sets, such as electronic health records and transcribed medical conversations”
So it trained on text that a healthcare provider extracted from the HPI/interview, labs/tests, radiology etc?
So it’s just a diagnostic assistant that can chat? Is it going to get conversational info out of non-English speakers and indigents? The urgently ill who are unable to vocalize what they’re feeling? The delirious who have “altered mental status”?
What about the huge amount of geriatrics who have significant cognitive decline and lack the ability and vocabulary to deliver information? And this is going to bother the terminally online - but what about the low SES who speak in virtually indecipherable urban patois?
This piece of shoftware isn’t going to help anyone. It’s going to be for articulate young to middle age educated people who don’t want to talk to a chatbot. And they’ll probably hate using it as they chat with it for filling out visit screening and new patient forms. If it ever deploys.
An “underserved” patient may use it out of desperation to get what they need but I would never in a billion years want this for a patient I gave a shit about.
I guess there’s the crux. They don’t give a shit. Some PE group will get it and you can talk to it before your virtual visit with your human doctor on Zoom. It’ll be like Elysium. Maybe I’m bitter because I still commute all over town. Maybe that’s what all you software people want, you like WFH so much, let the doctors do WFH too eh? Talk to the chatbot, you guys made it and it’s better than us medical cartel types anyways.
That being said, I had to laugh, this is one hyperbolic headline. There are of course some caveats.
"Few efforts to harness LLMs for medicine have explored whether the systems can emulate a physician’s ability to take a person’s medical history and use it to arrive at a diagnosis. Medical students spend a lot of time training to do just that, says Rodman. “It’s one of the most important and difficult skills to inculcate in physicians.”
This does take a lot of training and experience. The challenge in diagnosis isn't about asking a history and integrating the information, it's about effectively encouraging patients to provide necessary information that they might not realize is relevant or know how to articulate. Different patients often require wildly different approaches. Medical literacy does play a role, but a patient that would say, "Hi doctor, I experienced central chest pain accompanied by discomfort in the upper stomach that happened two hours ago" (from pg 33 of the preprint) is not realistic. More likely you get a vague complaint of "heartburn that started a while ago".
Similarly: "Currently, I'm not on any prescribed medications". More frequently you get something like "I take a blue one" or "Golly Telly" (Go-Lytely) or "Gabatini" (Gabapentin). I do think an LLM could probably parse these but such idiosyncrasies in a history do compound. And though someone may be prescribed a med and think they take it as prescribed, sometimes it takes a hunch and clinical experience to tease out that in fact the med is not being taken at all as indicated.
Moreover, the better bedside manner was assessed via text conversation. I wouldn't quite call that "bedside" manner. I also wonder how such a system will deal with patients that have self-diagnosed themselves and looked up the "right answers" to get what they want -- a difficult reality that takes some parsing to figure out what's real and what's not.
Overall though, the preprint, in contrast to the Nature news article/headline, does a better job of discussing these limitations. I congratulate them on excellent work. Thank god LLMs can't do surgery, though I'm sure my time will come as well.