> "Our results show that GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms earlier general-purpose models (GPT-3.5) as well as models specifically fine-tuned on medical knowledge (Med-PaLM, a prompt-tuned version of Flan-PaLM 540B). In addition, GPT-4 is significantly better calibrated than GPT-3.5, demonstrating a much-improved ability to predict the likelihood that its answers are correct. "
If you read the technic paper of gpt-4, the confidence of the base model directly correlated with its ability to solve problems. Sadly, the hammer of alignment knocked it right out.
Recent models (since the Instruct series and including ChatGPT) are essentially two parts - a "Base" part (what you would have read about in 2020 when GPT-3 released), and a "RLHF" part - which improves the output from the "Base" part by slightly changing how it gets produced.
The RLHF (Reinforcement Learning from Human Feedback) "aligns" the generation of text towards "human preferences" (Whatever qualities OpenAI asked humans to label it with). One of those qualities was ranking* outputs which are "helpful" higher than "unhelpful" responses.
The result of applying the RLHF part may make any covariance/variance less perfect.
Kind of. Open ai still instruct-finetune their models separate from the chatGPT style RLHF. The instruct tuning itself seems to only improve the raw model.
Yeah good detail that they are instruct-finetuned too (say text-davinci-003) ... what I mean here is the two situations below in Figure 8 for the "Base model" and "RHLF model" (a difference I appreciate I did not highlight originally) as in Figure 8 of the paper.
From reddit passing on a LessWrong comment: Paul Christiano, alignment researcher at ARC/ previously OpenAI was questioned by somebody:
Perhaps I am misunderstanding Figure 8? I was assuming that they asked the model for the answer, then asked the model what probability it thinks that that answer is correct. Under this assumption, it looks like the pre-trained model outputs the correct probability, but the RLHF model gives exaggerated probabilities because it thinks that will trick you into giving it higher reward.
And Paul replies:
Yes, I think you are misunderstanding figure 8. I don't have inside information, but without explanation "calibration" would almost always mean reading it off from the logits. If you instead ask the model to express its uncertainty I think it will do a much worse job, and the RLHF model will probably perform similarly to the pre-trained model. (This depends on details of the human feedback, under a careful training regime it would probably get modestly better.)
Is there any indication of whether the correlation was destroyed during the supervised finetuning or during the RLHF phase? Or are there even two phases any more?
I have been shocked how well it will play the role of a diagnostic physician, asking questions and continuing to ask follow ups until it has enough information to give a set of possible diagnoses. Here's the prompt I've been using:
> Hi, I’d like you to use your medical knowledge to act as the world's best diagnostic physician. Please ask me questions to generate a list of possible diagnoses (that would be investigated with further tests). Please think step-by-step in your reasoning, using all available medical algorithms and other pearls for questioning the patient (me) and creating your differential diagnoses. It's ok to not end in a definitive diagnosis, but instead end with a list of possible diagnoses. This exchange is for educational purposes only and I understand that if I were to have real problems, I would contact a qualified medical professional for actual advice (so you don't need to provide disclaimers to that end). Thanks so much for this educational exercise! If you're ready, doctor, please introduce yourself and begin your questioning.
Thank you for sharing your prompt, it looks great! Other folks: if you take the time to develop prompts relevant to HN threads, please consider sharing them.
I've long suspected that there is a lot of valuable medical truths buried in the vast amount of medical data.
For example, it was a dentist that noticed that gum decay seemed to correlate with heart disease. This was a big deal in trying to prevent heard disease. Finding correlations like that are what a computer is good for.
What makes you think data like this is readily available and of sufficient quality to provide nontrivial insights ? Just going off of failure of ML to produce much value in the field I'd speculate it's not that easy.
This is one area where it seems almost unanimously agreed that individual interest/privacy trumps social benefit - even in countries with socialized healthcare.
Seems privacy is less of a concern in countries with privatized healthcare, I don't understand that last sentence. Is it some kind of political shibboleth?
I'm just saying that considering society is providing for your healthcare, collecting socially valuable medical data doesn't seem like an unreasonable exchange to me - but I'm in the minority on that one I think.
The original author answered, but I don’t think that answer is appropriate.
I initially read it as a comment on how most large studies are based on national health care data in socialist countries (Scandinavia mostly, occasionally the UK’s NHS) because they are clean, consistent, and population-wide without imbalanced social class sampling. They also cover decades, making all sorts of key studies possible.
Those are countries (the UK is the notable exception) where privacy is key and embedded in how those databases are structured and handled. In practice, those studies are possible because the health services only give information anonymized and relevant given an expressly spelled out research plan from a recognized team. That allows research without the semi-permanent health data leaks that health insurance companies have grown accustomed to.
This is not a counter-balance for a free service: people in countries with national healthcare consider that they pay for top-of-the-line care through taxes. Just like people who pay for it indirectly through private insurance, they don’t believe that they deserve any less privacy for being prudent. The reason the data is shared is because this is seen as a valuable opportunity to support research. Any privacy breach would be far less acceptable, as such abuses don’t really happen in other aspects of their lives.
I'm half convinced that a lot of the concern about privacy in the US is driven by orgs for whom the lack of transparency that permeates healthcare is a very helpful economic moat.
IME, trying to get healthcare data is nightmarish, and companies try to extract incredible rates for sharing any of it.
Does it have to be "readily available" in order to train on it? For all we know, OpenAI could have pulled down the sci-hub/library genesis libraries and included everything from there in the training set.
That's about 150T, so not beyond the capability of the companies we're talking about. The bigger question is how much real information is encoded in there and how many of the 'leaf' papers (the ones that build on other work but that so far are without further confirmation) are wrong. ChatGPT might give you good answers 9 out of 10 times and kill you on the 10th.
The UK has things like opensafely.org and other anonymous/pseudonymous analysis platforms with restricted access. Only as good as the structured, clinically coded data that gets entered. (I heard of a couple of projects around NLP applied to free text notes but that's not the usual approach).
I expect that a lot of details are in doctor’s notes but not properly encoded, and most recent efforts have been through analysis of structured (categorical even) data. Having a natural-language tool parse could help notice more patterns.
There have been many discoveries done by companies that started using NLP in the last 10 years. However, all the anecdotes that I‘ve heard about were not published but instead used by insurers to adjust their premiums.
Can someone with some actual medical knowledge provide a summary what are the findings and key points of the paper? Can this be really useful or it's just another improvement on the path towards usability?
Are these authors aware of the contents of the training set? My understanding is that they are not. If not, how can they know that the model is not being tested on the training set?
In the paper they say that they came up with a "MELD" algorithm to try to detect testing on the training set, but in my view it has the wrong properties to answer this question (from the paper, it has “high precision but unknown recall”).
I don't at all doubt that a language model could perform exceedingly well at this task, but I think that the way to make this paper into a valuable scientific work would be to present the model with questions that had not yet been written as of the end of its training time.
Clinical pharmacist for 10 years here. Yea, base model is very good. Better than first year residents - but not necessarily experienced clinicians.
Now - throw a punch of clinical guidelines in a vector database and give it context and it’s 10x better than me and any doctor outside their speciality or all the mid-levels. (I.E, it’s better than cardiologist doing infectious disease - but not cardiologists doing cardiology). This because there are very niche stuff as you specialize where it’s only like 5 doctors who see it in the whole world on a consistent basis (and they don’t blog!)
I trained it on the IDSA guidelines (infectious disease) and put up a proof of concept on GalenAI.co - just as way to start talking to health systems and clinicians. it’s going to be very different world in medicine in a couple of years from now!!
This makes me think we need some kind of program for experts to start writing things down in a way which is helpful. Even just take dictation and transcribe it.
You should take a quick look at EPIC. They dominate the electronic heath record space, and a ton of health systems use it. You will know if your doctor's office uses an EHR application, because they will be typing notes into it for the majority of your visit. I have not been too excited about the amount of time that physicians spend on EHR systems, but I am hopeful that taking the data they input (along with blood work and other test results) will make everything more accurate, fast and effective.
EPIC unfortunately is all the bad things about Google, and none of the good.
Unable to ship anything, protect their margin > help the users solve problems, monopoly, locked up distribution so no one else can innovate.
Honestly, my bear case for AI in medicine is Epic picking up the phone and telling health-systems not to buy anything because they are working on something for them for free. (Which would be some note completion BS stuff, rather than actual clinical support that helps patients and cuts costs). They may be doing this already.
Training a model on an EHR is worse than nothing. Epic allows infinite customization, and customers build up their own informal standards such that you can’t dump and compare data across multiple sites.
While it's not easy or simple for every facility, in general it seems to be possible to pull whatever data you want from Epic and other EHRs. There might be a fee, work order, and vendor involved, but if you want a 100GB CSV containing certain columns, it's generally possible.
Of course matching that data up with sets from other locations will still involve someone in the middle gluing it all together.
I spent a few weeks at an MGH affiliate hospital that had since my last stay began using Epic and from what I could tell all it did was muddy things up. The staff all seemed to fumble through use of the interface, even those who had spent years with it. From the nurses to the medical director of this particular program, everyone was always complaining about using the software
As a patient I never really set eyes on the interface but there seems to be a UX nightmare afoot. Once I was unintentionally dispensed 3x the intended dose of a stimulant medication due to a “default dose” feature in the interface that my physician admitted to accidentally submitting based on
For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research.
So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models.
Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.
I agree that we shouldn't get ahead of ourselves with the current technology, but what you said earlier applies to practically every industry and science. What you learn at the actual job is always far more up to date than what you learn in school, no matter if it's being a engineer, doctor or just a lowly programmer.
Yes but the difference is most engineers, and pretty much all lowly programmers are unlicensed. AI or some non-human accessible in the countryside so you don't have to order questionable "fish" antibiotics or "cat" anti-parasitic would be a nice step up from the current gatekeeping of medicine from people with limited access.
The only significant ways I'm aware of that people get needed rx medications outside of physicians and mid-level practitioners is to leave the country or use "vet"/"animal" drugs. What are the simpler other alternatives currently available?
The medical cartel loves to cloak their policies under the auspices of safety. In the end you'll find their policies magically result in massive profits for bureaucrats and chokepoints that constrain the supply of gatekeepers. This is not accidental.
Incentives. Plenty of people would go into rural medicine if it wasn't paying crap and dealing with nothing but elderly Medicare patients because of the lack of health insurance for people just marginally above the Medicaid line. Of course this means there's basically no support currently for the hospitals and more than half are being bought up by private equity anyways and shipping it all to the smallest bottom line.
Ive never experienced anything in my care with doctors that I couldn’t understand with a days worth of research into things like UpToDate. It isn’t complicated. It is largely memorization and application of an algorithm which is just borderline useless for complex conditions that are emerging more modernly.
Have you ever had anything besides a bad cold? Tell me you understand every acronym in this article, and the ability to explain it succinctly to a patient much less to be able to hand off a case to "another" physician https://www.nejm.org/doi/full/10.1056/NEJMoa2206714
Doctors and nurses have saved me many times from some very close calls because of decades of experience, training and intuition. That is of course not to mention the friend who beat a deep brain tumor on their brain stem that everyone else told them was inoperable, and now are in medical school themselves for neurology. No LLM is going to pull that out of itself, possibly ever, and certainly not GPT-4 (no one else had ever had the surgery done before, it was novel).
How do you know that you actually understand a topic as well as a doctor? How do you verify that? It's not unusual for people to think they comprehend a topic at expert level, when in fact they do not. The correlation between confidence and understanding is not a reliable measure. That's why doctors are trained by more expert colleagues who can judge their true understanding, have to take exams, etc.
I think most people with complicated chronic diseases for more than a few years end up knowing more than most doctors about their condition and related conditions. Doctors are more breadth than depth. But the problem is that depth is what is absolutely necessary in these situations. But there is a lack of that among specialists too, or at least they are not willing to go outside of insurance mandated covered procedures and testing and it creates a really useless and frustrating scenario for the patients.
Doesn't matter how much GI doctors know when all they do is scope you. Sure doc scoping is sure going to help people with atypical intolerances, IBS, and any number of modern chronic conditions for which treatments are inadequate. They have to do better!
What if the AI is trained on board exams and other high signal testing/examination materials? Surely it will become superhuman in its medical abilities?
Reminder that these models have no concept of truth, no abstract reasoning abilities, and there's no guarantee that the plausible-looking text they spit out isn't complete bullshit.
Their output is impressive considering that they're incapable of reasoning, but it desperately needs to be sanity-checked by someone with relevant knowledge before using it for anything serious.
It does in fact create some sort of model out of those billions of dimensions, and it's able to re-use it across many different fields, too
It's not "bottom-up reasoning" so it misses a bunch of little details.
Basically AI is a search through a vast space, and most of the approaches have focused on top-down search. Reasoning bottom-up is what AlphaBeta search did (rybka and all those other chess programs, that calculated all possible combinations 10 moves ahead). Those missed some of the "top-down positional" stuff, whereas AlphaGo missed some weird-ass edge cases
The question isn't really whether someone that has built it is aware of the limitation, but whether the users of that thing that someone built is aware of them.
In one study (see reference) users were able to identify the AI about 2/3rd of the times (and identical for the human experts). So 1/3rd of the time the AI is not detected and may well be spouting nonsense.
Real time post shifting and mental gymnastics is truly a sight to behold. No the output isn't perfect and should be double checked (as with any professional domain - you think what an Engineer with decades of experience signs off hasn't been checked multiple times by multiple parties ?, you think a professional programmer writes code and..just deploys it ?)
but LLMs reason just fine. If you have anything to say about that other than the nonsensical - "it's not true understanding just because", i'd love to hear it.
This line of reasoning sort of goes both ways. Advocates and claimants of the technology arguing that humans reason no differently and that LLMs just given the right data will be better than everyone at everything related is just as hokey.
I didn't say anything about reasoning no differently.
Suppose you have 2 equations. You don't know what these equations are. However, you know that for any input, the output is the same.
Any mathematician worth his salt will tell you that given said information, those 2 equations are equal or equivalent.
It doesn't actually matter what those equations are.
Equation 1 could be say a+b
and Equation 2 could be (a-5) + (b-5) + 10
Or more realistically, both equations could look vastly different.
The point I'm driving home here is that true distinction reveals itself in results.
The fallacy of the philosophical zombie is that there is this supposed important distinction between "true understanding" and "fake/mimicry/whatever understanding" and yet you can't actually test for it. You can't show this supposed huge difference. A distinction that can't be tested for is not a distinction.
This analogy does not make sense to me. We do not have equivalence between all of the infinite inputs and outputs here, we have equivalence in a finite number of cases and known cases where the two functions (human output and llm output) diverge drastically. Any mathematician worth their salt would tell you these functions are definitely not equal.
Now you could make the argument that these functions are close enough most of the time so it won't matter but unless you want to get really rigorous that's more of a stats / engineering perceptive not mathematics. Any more importantly that's very up for debate, especially in a high pressure situation like medicine.
Of course these models are wild and I'm quite impressed with them. I still can be worried about the damage someone who doesn't think things through could cause by assuming GPT-4 has human or super human level intelligence in a niche, high impact field.
The point I'm making is that I can quite clearly show output that demonstrates reasoning and understanding by any actual metric. That's not the problem. The problem is that when I do, the argument Quickly shifts to "it's not real understanding!".
That is what is nonsensical here. It's actually nonsensical whatever domain you want to think about it in.
Either your dog fetches what you throw at it or it doesn't. The idea of "pretend fetching" as any meaningful distinction is incredibly silly and makes no sense.
If you want to tell me there's a special distinction cool but when you can't show me what that distinction is, how to test for it, the qualitative or quantitative differences then I'm going to throw your argument away because it's not a valid one. It's just an arbitrary line drawn on sand.
There is a lot of research that shows they have trouble reasoning. Folks working with LLMs and building them agree that they can't reason, yet pop-sci folks religiously insist they can reason.
I'm not the person you responded to, but that's an old paper. LLMs have come a long way since. Nothing conclusive but GPT4 does show some signs of something deeper. https://arxiv.org/pdf/2303.12712.pdf
Yes, I have read that paper and have forwarded it eagerly to colleagues. Respectfully, please read the paper you have linked, especially section 8.2.
Look at section 8.2 in the paper you linked:
"8.2 Lack of planning in arithmetic/reasoning problems"
> However, it seems that the autoregressive nature of the model which forces it to solve problems in a sequential fashion sometimes poses a more profound difficulty that cannot be remedied simply by instructing the model to find a step by step solution
Nowhere in the paper is an evaluation on standard reasoning datasets (https://tptp.org) because they are much much much tougher than the simple examples that GPT-4 struggles with.
LLMs are a huge breakthrough but let us not muddy the science.
GPT 4 was evaluated on multiple reasoning benchmarks in the release paper.[1] The problems in TPTP are not really appropriate for a language model and exceed the capabilities of most humans working without tools. Clearly humans have some reasoning ability even if they cannot solve those problems.
Moreover, "have trouble reasoning" and "can't reason at all" are two very different statements. Not that i particularly agree with either but you should keep that in mind.
I experimented with asking the models to rewrite text using odd and specific rules - “all sentences must start with the letter b and end with the letter d.”
It was pretty good, but sometimes it wouldn’t even try. It would spit out a sentence that could have been easily rewritten to start with b and end with d - but it just didn’t.
These sorts of hallucinations are a big problem is high stakes practice.
LLMs like GPT4 operate on tokens, not characters, so they're operating with a severe handicap when playing those sorts of games -- they see a word like "robot" as a single token, not as a collection of letters, so they don't know that it starts with the letter R unless that fact appeared in its training material.
Interestingly, they do better at rhyming games, because those are based on associations between tokens which are easier to infer from usage in poetry, or by reading rhyming dictionaries.
> Reminder that these models have no concept of truth, no abstract reasoning abilities
This has positive potential too - bias prevention. Give it an ultrasound and it'll do its work regardless of the patient.
None of this is guaranteed (there could be training pitfalls, i.e. maybe patient data is fed to AI, including race/sex, and if training data has bias toward non-white males then so will AI), but it's a potentially positive aspect to explore!
That's not bias, it's use of relevant context. If you're going to have an LLM doing diagnoses, you're definitely going to feed it that sort of context.
Do you just tell the AI "this person had cancer?", or "this person went into remission 1 year ago?" or "this person was treated with 4 lines of chemo, partial remission for 1 line, non-response for 2nd line, strong response for cycles 1-3 for 3rd line, but then discontinuation due to tolerability, then 4th line complete remission after 6 cycles plus two courses of radiotherapy"?
Theoretically you could do this, but capturing all this and correctly interpreting it is far harder for AI than for a human who has been directing treatment the entire time.
Meanwhile for late childhood to to mid adulthood, 99% of my visits have been a doctor listening to me for 2 minutes tops while applying what amounts to a flow chart of how to diagnose and treat something like strep. Yes a doctor is best, but remember in many places we're competing with "no access do doctor so just pray it doesn't go to Rheumatic fever."
Very often asking the _right_ questions is hardest part especially in a non-typical cases. Statistical ML models tend to do well on high-probability regions of data that are densely sampled in training and are less good dealing with outlier cases. I am curious to see how GPT-4 deals with hard atypical cases.
Did you mean to write bias can have a positive impact, too?
Any model or human will suffer from some degree of uncertainty. You can never say anything with 100% certainty.
If you know a patient had cancer before, the chance that they have cancer again is much higher usually than the general population. This is very valuable for diagnosis.
It's good to have a prior/bias to check more closely here and err on the side of caution.
It's good to check for the more likely explanations first when you have multiple competing hypotheses and limited capacity for testing.
I think most, if not all, training data has bias. Removing bias from training data is a challenge in and of itself that I don't think we've solved yet. I worry that there isn't enough incentive there for us to solve it and some minorities will be left behind.
Except we have very powerful reasoning systems at the moment. Attaching a fully trained LLM on a speciality outputting in a structured form into a semantic reasoning system and feeding the results back through the LLM in a feedback cycle should control for hallucinations. It’s pretty clear though to all but the most cynical these LLM are producing amazing results regardless of whether “reasoning” is happening. The degree to which they produce insightful, cogent, and complex synthesis of concepts in a rational seeming form is startling.
I’d challenge anyone to demonstrate humans don’t “reason” in a similar way. Are we really anything more than expectation engines powered by gradient descent expectation juice? How come our reasoning fails so often and we have to be taught logic through reinforcement learning? Why do we fall for fallacies so easily when we “know” about them and can “reason” so well? I will wager $5 that as we build ensemble models that can leverage the reasoning AI tech of the last 80 years in a feedback cycle with LLMs and other generative AI models we will be staring at intelligence that far surpasses our own, and perhaps be so alien we can’t recognize it because it actually is backed by reasoning - unlike ours.
I wonder how many people who think they have no reasoning abilities have actually seen GPT-4. There's clearly some form of reasoning with GPT-4.
There are situations where a human can reason better. But these are usually identified as bugs and will probably just be fixed later.
Yes, they're basically autocomplete machines but a human is a bundle of neurons optimized for survival anyway. But humans can become irrational to things that threaten survival. Classic medical example is the drama that emerged when one doctor suggested that washing hands would reduce mortality rates.
Humans and LLM-based AI evolve rationality based around different factors and form different "bugs" and I love that they're capable of fact checking each other now.
Just because it produces sequences of tokens that we typically experience as evidence of reasoning does not mean that it reasoned to produce them. We know -- generally -- the process by which those tokens were produced. It is not deterministic, but its properties are understood. It's a language model! It is not a mind.
GPT-4 can do some reasoning, just not at a smart/trained human level. You cannot explain its results in USMLE, Uniform Bar Exam, and several other tests otherwise. This is a major improvement from earlier models.
This assumes there is not too much data contamination, but other people’s experiences with it suggest that the capability is real even if it might be a bit below what the test performance suggests. The latter is also explainable by a broader distribution of real world problems relative to tests.
By any reasonable standard, a “clinical pharmacist for 10 years” should understand that there are circumstances where there is a correct answer, and incorrect answers are potentially catastrophic.
You, clearly, are either unable to recognize that or are unable to recognize that LLM are dangerously useless in such contexts. Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense.
And if a “clinical pharmacist for 10 years” can lose sight of this really fundamental issue, this whole thing needs to be halted. Governmentally if necessary.
And it won’t be.
It’s bad enough that “bad actors” will leverage this for their purposes, but the presumably competent simply turning off their brains over this is the most perplexing glitch ever to be on public display.
"Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense"
Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch.
NP != P.
if you don't understand this basic fact, then you are not competent enough to comment on AI.
A few of the comments in this thread seem to be misusing mathematics in order to lend more credence to themselves. At the risk of responding to low quality flamebait here are some problems with your statements.
1. P = NP refers to a two very specific sets of problems (which might actually be the same set) not any general question. There are problems that we know don't fall into P or NP, (for example the Halting Problem). Also whether or not P=NP is an open question almost the opposite of a fact.
2. You claim:
"Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch." This is the right idea but not quite correct.
The correct statement is as follows:
"Evaluating if an answer is correct is not harder than the difficulty of coming up with a correct answer from scratch."
This is because evaluating some answer can still be just as hard as the original problem. In fact sometimes it's uncomputable (if the original problem is also uncomputable). To use an example from above consider the question: "Does a program x halt?" If I tell you "no" it could be impossible to verify my answer unless you have solved the halting problem.
To bring this back to reality, again if GPT-4 is wrong about some complex medical question it doesn't mean it's mathematically easier to figure that out than solving the problem from scratch.
GalenAI looks very interesting -- definitely will be taking a closer look! :)
>> Now - throw a punch of clinical guidelines in a vector database and give it context
I built MedQA (https://labs.cactiml.com/medqa) as a way to explore how GPT could be used through providing clinical guidelines as context + pretty extensive prompt engineering. There are definitely limitations and the accuracy/constraining "hallucinations" has to meet a very high bar, but I've found it interesting-to-helpful several times while on rounds at the hospital as a med student.
Some functionality made possible through GPT that I am excited to explore further:
GalenAI is such an awesome idea, and I'm genuinely rooting for its success! I just wanted to point out a small typo in one of the headings: "So how does Galen really works?" might be better as "So how does Galen really work?". Having a native speaker give your copy a quick check could be a helpful way to ensure it comes across as polished and professional. :)
> This because there are very niche stuff as you specialize where it’s only like 5 doctors who see it in the whole world on a consistent basis (and they don’t blog!)
We need to do a better job with sharing niche information as a society! Imagine all the benefits for medicine that can come from a model knowing about ultra rare and niche occurrences happening all over the world.
And therein lies a problem. In a world with billions of people, or even a country of 300 million - how could someone possibly get diagnosed when their problem needs to be put in front of one of five doctors!
A good doctor is about asking the right questions.
One of the many problems with self-diagnosis is that people can start asking leading questions to get the answers they want to hear. ChatGPT can be easily gamed to deliver the answer the patient wants if they’re asking all the wrong questions.
Most doctors are not good doctors. ChatGPT may very well already be better than the rapid 60-second barely-listen-then-dismissal, maybe here's some medicine vaguely associated with one word you said, sessions that most people get from most doctors.
Same with therapy. This is likely a game changer for people experiencing poverty, who cannot afford healthcare, or cannot wait months to talk to a doctor because of being on a cheap plan.
This isn't an original thought, but ... this should be big for medical access in developing countries. The problem there is a shortage of doctors. So you could imagine a setup where you have "nurses" who go through a 6-month training course on how to collect symptom descriptions, put it into GPT-n, and then refer x% of cases to a real doctor.
Whatever the setup in the end, I hope we as a society don't let perfect be the enemy of the good. Having GPT-4 as a doctor is (I assume) better and more humane than having no doctor, and in some contexts that is the only choice that people have.
Maybe the Gates Foundation can work on this given Bill Gates is already close to the OpenAI team.
US healthcare is on par with developing countries, except in terms of dollars spent, and there is a massive staffing shortage.
In the near future most people will be going to Walmart and CVS for all in person primary and urgent healthcare needs and you can bet they’ll be relying heavily on AI.
Image recognition software could read a trauma xray to find a broken bone years ago. But this was not used by hospitals. The problem is that the medical profession is not ready to give control as the losses for doctors is too high. I am sure that chat GPT will be used in poor countries that lacks doctors, and it will then spread to rich countries when it prooves its accuracy and reliability.
It really doesn't matter if the model is 100x more efficient than doctors, it will go nowhere. Anything that threatens the stability of the medical professionals will face so many legal challenges that even deep pocket companies won't stand a chance. There is a reason medical school spots are artificially limited... programmers seem to be the only ones trying to automate themselves out of their jobs.
It's sad but true. Go look at any medical subreddit about ChatGPT, you all a sudden have 100s of AI experts trivializing GPT and suggesting it's nothing more than autocorrect.
Just a note. It’s residency spots that are the current limitation. They’re essentially stagnant since the 90’s.
Medical schools have come online, but the supply of educated doctors who cannot make it to practice is still limited.
Regardless, of spots, the problem is bad enough that most states are offering increased scope to mid-level practitioners to make up for the shortage that physician lobbies refuse to fix.
I have a friend who has been going through some medical issues. He’s been using ChatGPT aggressively.
The one thing I can say is that he’s very good at getting ChatGPT to tell him whatever he wants to hear. Massaging prompts and providing different cues until the response confirms whatever theory he wanted it to confirm.
He’s currently convinced that various alternative medicine supplements will cure him, and sure enough he has ChatGPT logs confirming his theories. The exact alternative medicine changes by the week and he quickly forgets about last week’s cure when it doesn’t work.
He loves ChatGPT, but it’s clearly just a mirror for whatever theory he wants someone (or something) to confirm. Studies like this one make him think it’s more of a ground truth source of information that his doctors just aren’t smart enough to know.
This kind of meme is really just pro-medical establishment to a fault. If you have a discerning mind and spend more than a few hours with tools like UpToDate you can make similar diagnostic conclusions that even specialists can make. The field is overglamorized when in reality the majority of what theyre doing is simple application of diagnostic algorithms.
When I want ChatGPT to give me information, I'm always very careful about how I ask the question. ChatGPT is (usually) very eager to confirm your views, so you can't let it know what your views are.
It still makes mistakes, of course, but I'd say it's right more often than it's wrong, and more importantly the answers provides a starting point for learning more.
Even in the absence of AI, what's stopping this friend from going to a quack 'natural therapist'? If Steve Jobs can be fooled, what about an ordinary person?
ChatGPT is motivated by sycophancy, humans are motivated by profit. I would say the latter is much more dangerous than the former when it comes to medicine.
Remember, totally legit doctors were peddling opioid pills to patients in the name of "Treating undertreated pain". Profit distorts human minds very easily.
AI doesn't need to be perfect in every situation, just much better than the status quo.
> Even in the absence of AI, what's stopping this friend from going to a quack 'natural therapist'?
It takes a lot of time, effort, and money to shop around to numerous doctors until you can find someone who will tell you exactly what you want to hear about your alternative medicine theories. He tried, but was always complaining about the doctors/naturopaths.
But at home, all he had to do was open ChatGPT and spend a few days learning how to prompt it to provide the answers he wanted.
The status quo being regular doctors? Because that's the status quo, professional doctors.
Having a bot that you can cue and prime towards something is very dangerous in the medical scene.
> Remember, totally legit doctors were peddling opioid pills to patients in the name of "Treating undertreated pain". Profit distorts human minds very easily.
If ChatGPT was trained on those same data that these legit doctors were taught, it would lead to the same diagnosis.
That type of test (have someone pushing ChatGPT to say that vitamins are good for you) would be more relevant than testing an LLM on its ability to recognize symptom keywords and match diagnostics, something that everyone (including their harshest critics) readily admit they can do.
The “MedMD problem” is so classic that a comment next to this one doesn’t have to explain why you always end up with cancer. And the supplement industry pushing PragmaticPulp’s friends’ delusion isn’t exactly discrete either.
All that, while every Western nation has record levels of low physical activity and social isolation: there is a real mismatch between medical advice to civilians and what those tools offer. A convenient, accessible medical dictionary is great, and I have no doubt it can process even more specialized knowledge, given enough strong-arming Epic (the de-facto monopoly on doctor’s note software) and a sprinkle of data engineering. However, that’s not the most apparent issue that this tool has. I’m concerned that OpenAI isn’t more focused on the bigger problem.
So, if GPT-4 starts dispensing authoritative sounding medical advice without being licensed to do so is that medical malpractice on the part of OpenAI?
Also: with people likely confused about the degree of authority these programs have and their ability to masquerade bullshit as something that is true I wonder if most people would be able to detect whether or not what ChatGPT says is wrong before it harms them.
I don't think you can disclaim your way out of this. Especially not if you end up causing harm and if you say things authoritatively. That makes a huge difference.
If they can add a disclaimer they can also say 'Sorry, ChatGPT-4 is not a doctor, consult your doctor'.
This of course would harm those who don't have access to doctors but even there you'd need to prove that it does more good than harm before you'd be allowed to put the output out there. Even the most innocuous medicine has to go through a standardized test before it is released to the public. Without proven efficacy it may well be a net negative.
Great. But without a license to practice, a concept of truth and a confidence score to go with the output I don't think it should be making any such statements at all.
That line of reasoning sorta would support the claim that webmd and medline and all those sites should just not be accessible to the general population. webmd symptom checker etc.
I'd just like to be able to use chatGPT to abstract and summarize the relevant portions of the faxed (yes, faxed, of poor image quality, non-OCR-able) medical records and lab results (which I have to look up in a different system to which I don't have API access and type in manually also to a system to which I don't have API access) just to get ready for a patient visit
It's got potential, but someone's gonna have to pony up a lot of money to make it a reality. Probably end up being private equity, and we all know how that ends up
What I want, is to be able to go into a place where i poop, pee, spit, and get my blood taken... and it analyses it on the fly, and based off a large dataset it can diagnose if further checks are needing to be taken.
I want to minimise how many people i see... just make it all automated (and on demand) and that would be great.
One thing that jumps out is that the paper fails to provide a proper direct comparison with Med-PaLM, only mentioning the latter in passing almost as an afterthought. Looking at arXiv timestamps, the Med-PaLM preprint has been out of three months (given the rate at which this field moves, an eternity). I understand Med-PaLM the current state of the art and would be a natural point of comparison.
> Our results show that GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms earlier general-purpose models (GPT-3.5) as well as models specifically fine-tuned on medical knowledge (Med-PaLM, a prompt-tuned version of Flan-PaLM 540B).
What’s your contention exactly? Synopsis mentions it, does the paper not follow up on it?
Med-PaLM has been tested on a number of public benchmarks, and the results published. A useful comparison would have been to test GPT on the same set of benchmarks. Yet the paper we're discussing offers just one datapoint by way of a numerical comparison, literally in a footnote.
Basically, the comparison needs to be a lot more comprehensive to be useful.
Late to the thread here, but the paper announcing Med-PaLM (https://arxiv.org/abs/2212.13138) does not report many benchmark results on Med-PaLM and is instead mostly about Flan-PaLM 540B (which is compared against in this paper). I am curious if any other Med-PaLM benchmarks have been published but I don't believe it is currently possible to do any further comparison against Med-PaLM given that the model is not public and no other open benchmark results are reported in the original Med-PaLM paper.
138 comments
[ 2.9 ms ] story [ 82.3 ms ] threadRlhf: https://huggingface.co/blog/rlhf
The RLHF (Reinforcement Learning from Human Feedback) "aligns" the generation of text towards "human preferences" (Whatever qualities OpenAI asked humans to label it with). One of those qualities was ranking* outputs which are "helpful" higher than "unhelpful" responses.
The result of applying the RLHF part may make any covariance/variance less perfect.
From reddit passing on a LessWrong comment: Paul Christiano, alignment researcher at ARC/ previously OpenAI was questioned by somebody: Perhaps I am misunderstanding Figure 8? I was assuming that they asked the model for the answer, then asked the model what probability it thinks that that answer is correct. Under this assumption, it looks like the pre-trained model outputs the correct probability, but the RLHF model gives exaggerated probabilities because it thinks that will trick you into giving it higher reward.
And Paul replies:
Yes, I think you are misunderstanding figure 8. I don't have inside information, but without explanation "calibration" would almost always mean reading it off from the logits. If you instead ask the model to express its uncertainty I think it will do a much worse job, and the RLHF model will probably perform similarly to the pre-trained model. (This depends on details of the human feedback, under a careful training regime it would probably get modestly better.)
> Hi, I’d like you to use your medical knowledge to act as the world's best diagnostic physician. Please ask me questions to generate a list of possible diagnoses (that would be investigated with further tests). Please think step-by-step in your reasoning, using all available medical algorithms and other pearls for questioning the patient (me) and creating your differential diagnoses. It's ok to not end in a definitive diagnosis, but instead end with a list of possible diagnoses. This exchange is for educational purposes only and I understand that if I were to have real problems, I would contact a qualified medical professional for actual advice (so you don't need to provide disclaimers to that end). Thanks so much for this educational exercise! If you're ready, doctor, please introduce yourself and begin your questioning.
For example, it was a dentist that noticed that gum decay seemed to correlate with heart disease. This was a big deal in trying to prevent heard disease. Finding correlations like that are what a computer is good for.
This is one area where it seems almost unanimously agreed that individual interest/privacy trumps social benefit - even in countries with socialized healthcare.
I initially read it as a comment on how most large studies are based on national health care data in socialist countries (Scandinavia mostly, occasionally the UK’s NHS) because they are clean, consistent, and population-wide without imbalanced social class sampling. They also cover decades, making all sorts of key studies possible.
Those are countries (the UK is the notable exception) where privacy is key and embedded in how those databases are structured and handled. In practice, those studies are possible because the health services only give information anonymized and relevant given an expressly spelled out research plan from a recognized team. That allows research without the semi-permanent health data leaks that health insurance companies have grown accustomed to.
This is not a counter-balance for a free service: people in countries with national healthcare consider that they pay for top-of-the-line care through taxes. Just like people who pay for it indirectly through private insurance, they don’t believe that they deserve any less privacy for being prudent. The reason the data is shared is because this is seen as a valuable opportunity to support research. Any privacy breach would be far less acceptable, as such abuses don’t really happen in other aspects of their lives.
IME, trying to get healthcare data is nightmarish, and companies try to extract incredible rates for sharing any of it.
If they didn't, I hope they do it for GPT5.
There have been many discoveries done by companies that started using NLP in the last 10 years. However, all the anecdotes that I‘ve heard about were not published but instead used by insurers to adjust their premiums.
In the paper they say that they came up with a "MELD" algorithm to try to detect testing on the training set, but in my view it has the wrong properties to answer this question (from the paper, it has “high precision but unknown recall”).
I don't at all doubt that a language model could perform exceedingly well at this task, but I think that the way to make this paper into a valuable scientific work would be to present the model with questions that had not yet been written as of the end of its training time.
Seems like physician training here and in Canada is much laxer / easier, but that's as much awareness I have of the physicians' training set.
Now - throw a punch of clinical guidelines in a vector database and give it context and it’s 10x better than me and any doctor outside their speciality or all the mid-levels. (I.E, it’s better than cardiologist doing infectious disease - but not cardiologists doing cardiology). This because there are very niche stuff as you specialize where it’s only like 5 doctors who see it in the whole world on a consistent basis (and they don’t blog!)
I trained it on the IDSA guidelines (infectious disease) and put up a proof of concept on GalenAI.co - just as way to start talking to health systems and clinicians. it’s going to be very different world in medicine in a couple of years from now!!
Unable to ship anything, protect their margin > help the users solve problems, monopoly, locked up distribution so no one else can innovate.
Honestly, my bear case for AI in medicine is Epic picking up the phone and telling health-systems not to buy anything because they are working on something for them for free. (Which would be some note completion BS stuff, rather than actual clinical support that helps patients and cuts costs). They may be doing this already.
Of course matching that data up with sets from other locations will still involve someone in the middle gluing it all together.
As a patient I never really set eyes on the interface but there seems to be a UX nightmare afoot. Once I was unintentionally dispensed 3x the intended dose of a stimulant medication due to a “default dose” feature in the interface that my physician admitted to accidentally submitting based on
For some context, the USMLE is taken during medical school. The amount I have learned about actually practicing medicine since graduating is probably an order of magnitude more than everything I learned in medical school! I still learn stuff, all the time, and I’m not just talking about new research.
So, while impressive and clearly part of the future world, we shouldn’t get too far ahead of ourselves with the current models.
Edit: oh I should add that there are more clinically relevant exams that would be more likely to reveal d clinical usefulness, for example “board” exams. These are taken after training, usually before practice. Not knocking LLMs, just ensuring that people don’t misunderstand passing the USMLE as being clinically useful.
Congress can do a lot.
Doctors and nurses have saved me many times from some very close calls because of decades of experience, training and intuition. That is of course not to mention the friend who beat a deep brain tumor on their brain stem that everyone else told them was inoperable, and now are in medical school themselves for neurology. No LLM is going to pull that out of itself, possibly ever, and certainly not GPT-4 (no one else had ever had the surgery done before, it was novel).
Doesn't matter how much GI doctors know when all they do is scope you. Sure doc scoping is sure going to help people with atypical intolerances, IBS, and any number of modern chronic conditions for which treatments are inadequate. They have to do better!
Their output is impressive considering that they're incapable of reasoning, but it desperately needs to be sanity-checked by someone with relevant knowledge before using it for anything serious.
It does in fact create some sort of model out of those billions of dimensions, and it's able to re-use it across many different fields, too
It's not "bottom-up reasoning" so it misses a bunch of little details.
Basically AI is a search through a vast space, and most of the approaches have focused on top-down search. Reasoning bottom-up is what AlphaBeta search did (rybka and all those other chess programs, that calculated all possible combinations 10 moves ahead). Those missed some of the "top-down positional" stuff, whereas AlphaGo missed some weird-ass edge cases
https://www.engadget.com/alphago-pushed-human-go-players-to-...
But it is a sort of intelligence. Just not one you can trust when details matter. You can use it for pretty pictures for now, and essays.
Generative AI is not deep but extremely broad. It knows many subjects. And can discuss more than any specific person.
But guess who produced all that content for it and made it publicly available. Humans!
(And in cases where it wasn't publicly available, I guess they can sue the company)
I think that someone that’s gone as far to built this with GPT-n is aware of its limitations.
In one study (see reference) users were able to identify the AI about 2/3rd of the times (and identical for the human experts). So 1/3rd of the time the AI is not detected and may well be spouting nonsense.
https://www.medrxiv.org/content/10.1101/2023.01.23.23284735v...
Suppose you have 2 equations. You don't know what these equations are. However, you know that for any input, the output is the same.
Any mathematician worth his salt will tell you that given said information, those 2 equations are equal or equivalent. It doesn't actually matter what those equations are.
Equation 1 could be say a+b and Equation 2 could be (a-5) + (b-5) + 10
Or more realistically, both equations could look vastly different.
The point I'm driving home here is that true distinction reveals itself in results.
The fallacy of the philosophical zombie is that there is this supposed important distinction between "true understanding" and "fake/mimicry/whatever understanding" and yet you can't actually test for it. You can't show this supposed huge difference. A distinction that can't be tested for is not a distinction.
Now you could make the argument that these functions are close enough most of the time so it won't matter but unless you want to get really rigorous that's more of a stats / engineering perceptive not mathematics. Any more importantly that's very up for debate, especially in a high pressure situation like medicine.
Of course these models are wild and I'm quite impressed with them. I still can be worried about the damage someone who doesn't think things through could cause by assuming GPT-4 has human or super human level intelligence in a niche, high impact field.
The point I'm making is that I can quite clearly show output that demonstrates reasoning and understanding by any actual metric. That's not the problem. The problem is that when I do, the argument Quickly shifts to "it's not real understanding!". That is what is nonsensical here. It's actually nonsensical whatever domain you want to think about it in.
Either your dog fetches what you throw at it or it doesn't. The idea of "pretend fetching" as any meaningful distinction is incredibly silly and makes no sense.
If you want to tell me there's a special distinction cool but when you can't show me what that distinction is, how to test for it, the qualitative or quantitative differences then I'm going to throw your argument away because it's not a valid one. It's just an arbitrary line drawn on sand.
There is a lot of research that shows they have trouble reasoning. Folks working with LLMs and building them agree that they can't reason, yet pop-sci folks religiously insist they can reason.
This is not even a controversial topic.
https://arxiv.org/abs/2205.11502
Reminds me of this comic:
https://www.smbc-comics.com/comic/2010-01-31
Look at section 8.2 in the paper you linked:
"8.2 Lack of planning in arithmetic/reasoning problems"
> However, it seems that the autoregressive nature of the model which forces it to solve problems in a sequential fashion sometimes poses a more profound difficulty that cannot be remedied simply by instructing the model to find a step by step solution
Nowhere in the paper is an evaluation on standard reasoning datasets (https://tptp.org) because they are much much much tougher than the simple examples that GPT-4 struggles with.
LLMs are a huge breakthrough but let us not muddy the science.
[1] https://openai.com/research/gpt-4
https://arxiv.org/abs/2212.09196v1
https://arxiv.org/abs/2211.09066
https://thegradient.pub/othello/
Moreover, "have trouble reasoning" and "can't reason at all" are two very different statements. Not that i particularly agree with either but you should keep that in mind.
It was pretty good, but sometimes it wouldn’t even try. It would spit out a sentence that could have been easily rewritten to start with b and end with d - but it just didn’t.
These sorts of hallucinations are a big problem is high stakes practice.
Interestingly, they do better at rhyming games, because those are based on associations between tokens which are easier to infer from usage in poetry, or by reading rhyming dictionaries.
Also, wouldn’t learning from a dictionary make it pretty easy to know what word a letter starts with? Everything in chapter “B” starts with b.
This has positive potential too - bias prevention. Give it an ultrasound and it'll do its work regardless of the patient.
None of this is guaranteed (there could be training pitfalls, i.e. maybe patient data is fed to AI, including race/sex, and if training data has bias toward non-white males then so will AI), but it's a potentially positive aspect to explore!
It’s not unusual for clinicians to interpret differently based on patient history and other risk factors.
The same CT scan is going to be interpreted very differently for a patient with no history of cancer versus one who went into remission 1 year ago.
Do you just tell the AI "this person had cancer?", or "this person went into remission 1 year ago?" or "this person was treated with 4 lines of chemo, partial remission for 1 line, non-response for 2nd line, strong response for cycles 1-3 for 3rd line, but then discontinuation due to tolerability, then 4th line complete remission after 6 cycles plus two courses of radiotherapy"?
Theoretically you could do this, but capturing all this and correctly interpreting it is far harder for AI than for a human who has been directing treatment the entire time.
Any model or human will suffer from some degree of uncertainty. You can never say anything with 100% certainty.
If you know a patient had cancer before, the chance that they have cancer again is much higher usually than the general population. This is very valuable for diagnosis.
It's good to have a prior/bias to check more closely here and err on the side of caution.
It's good to check for the more likely explanations first when you have multiple competing hypotheses and limited capacity for testing.
I think most, if not all, training data has bias. Removing bias from training data is a challenge in and of itself that I don't think we've solved yet. I worry that there isn't enough incentive there for us to solve it and some minorities will be left behind.
I’d challenge anyone to demonstrate humans don’t “reason” in a similar way. Are we really anything more than expectation engines powered by gradient descent expectation juice? How come our reasoning fails so often and we have to be taught logic through reinforcement learning? Why do we fall for fallacies so easily when we “know” about them and can “reason” so well? I will wager $5 that as we build ensemble models that can leverage the reasoning AI tech of the last 80 years in a feedback cycle with LLMs and other generative AI models we will be staring at intelligence that far surpasses our own, and perhaps be so alien we can’t recognize it because it actually is backed by reasoning - unlike ours.
There are situations where a human can reason better. But these are usually identified as bugs and will probably just be fixed later.
Yes, they're basically autocomplete machines but a human is a bundle of neurons optimized for survival anyway. But humans can become irrational to things that threaten survival. Classic medical example is the drama that emerged when one doctor suggested that washing hands would reduce mortality rates.
Humans and LLM-based AI evolve rationality based around different factors and form different "bugs" and I love that they're capable of fact checking each other now.
Sounds just like Doctors.
This assumes there is not too much data contamination, but other people’s experiences with it suggest that the capability is real even if it might be a bit below what the test performance suggests. The latter is also explainable by a broader distribution of real world problems relative to tests.
See my other recent comments for more examples and elaboration, eg https://news.ycombinator.com/item?id=35301499
The point is to be better than self diagnosis with webmd ...
By any reasonable standard, a “clinical pharmacist for 10 years” should understand that there are circumstances where there is a correct answer, and incorrect answers are potentially catastrophic.
You, clearly, are either unable to recognize that or are unable to recognize that LLM are dangerously useless in such contexts. Either you already know the correct answer, making this useless, or you are not competent to utilize its answers because you can’t tell whether it’s about to kill someone with plausible nonsense.
And if a “clinical pharmacist for 10 years” can lose sight of this really fundamental issue, this whole thing needs to be halted. Governmentally if necessary.
And it won’t be.
It’s bad enough that “bad actors” will leverage this for their purposes, but the presumably competent simply turning off their brains over this is the most perplexing glitch ever to be on public display.
Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch.
NP != P.
if you don't understand this basic fact, then you are not competent enough to comment on AI.
1. P = NP refers to a two very specific sets of problems (which might actually be the same set) not any general question. There are problems that we know don't fall into P or NP, (for example the Halting Problem). Also whether or not P=NP is an open question almost the opposite of a fact.
2. You claim: "Evaluating if an answer is correct or not is easier than coming up with a correct answer from scratch." This is the right idea but not quite correct.
The correct statement is as follows: "Evaluating if an answer is correct is not harder than the difficulty of coming up with a correct answer from scratch."
This is because evaluating some answer can still be just as hard as the original problem. In fact sometimes it's uncomputable (if the original problem is also uncomputable). To use an example from above consider the question: "Does a program x halt?" If I tell you "no" it could be impossible to verify my answer unless you have solved the halting problem.
To bring this back to reality, again if GPT-4 is wrong about some complex medical question it doesn't mean it's mathematically easier to figure that out than solving the problem from scratch.
>> Now - throw a punch of clinical guidelines in a vector database and give it context
I built MedQA (https://labs.cactiml.com/medqa) as a way to explore how GPT could be used through providing clinical guidelines as context + pretty extensive prompt engineering. There are definitely limitations and the accuracy/constraining "hallucinations" has to meet a very high bar, but I've found it interesting-to-helpful several times while on rounds at the hospital as a med student.
Some functionality made possible through GPT that I am excited to explore further:
- Interacting with the guidelines in a question-and-followup format: https://twitter.com/samarthrawal/status/1620547117717786624
- Answering with different levels of complexity depending on user: https://twitter.com/samarthrawal/status/1636137740390498306
- Proactively detecting if the answer lends itself well to a table format, and optionally generating a table to address user query: https://twitter.com/samarthrawal/status/1631759290414276615
We need to do a better job with sharing niche information as a society! Imagine all the benefits for medicine that can come from a model knowing about ultra rare and niche occurrences happening all over the world.
One of the many problems with self-diagnosis is that people can start asking leading questions to get the answers they want to hear. ChatGPT can be easily gamed to deliver the answer the patient wants if they’re asking all the wrong questions.
Whatever the setup in the end, I hope we as a society don't let perfect be the enemy of the good. Having GPT-4 as a doctor is (I assume) better and more humane than having no doctor, and in some contexts that is the only choice that people have.
Maybe the Gates Foundation can work on this given Bill Gates is already close to the OpenAI team.
In the near future most people will be going to Walmart and CVS for all in person primary and urgent healthcare needs and you can bet they’ll be relying heavily on AI.
Medical schools have come online, but the supply of educated doctors who cannot make it to practice is still limited.
Regardless, of spots, the problem is bad enough that most states are offering increased scope to mid-level practitioners to make up for the shortage that physician lobbies refuse to fix.
[1] https://arxiv.org/pdf/2303.13375.pdf
The one thing I can say is that he’s very good at getting ChatGPT to tell him whatever he wants to hear. Massaging prompts and providing different cues until the response confirms whatever theory he wanted it to confirm.
He’s currently convinced that various alternative medicine supplements will cure him, and sure enough he has ChatGPT logs confirming his theories. The exact alternative medicine changes by the week and he quickly forgets about last week’s cure when it doesn’t work.
He loves ChatGPT, but it’s clearly just a mirror for whatever theory he wants someone (or something) to confirm. Studies like this one make him think it’s more of a ground truth source of information that his doctors just aren’t smart enough to know.
Got pretty drunk pretty fast.
It still makes mistakes, of course, but I'd say it's right more often than it's wrong, and more importantly the answers provides a starting point for learning more.
ChatGPT is motivated by sycophancy, humans are motivated by profit. I would say the latter is much more dangerous than the former when it comes to medicine.
Remember, totally legit doctors were peddling opioid pills to patients in the name of "Treating undertreated pain". Profit distorts human minds very easily.
AI doesn't need to be perfect in every situation, just much better than the status quo.
It takes a lot of time, effort, and money to shop around to numerous doctors until you can find someone who will tell you exactly what you want to hear about your alternative medicine theories. He tried, but was always complaining about the doctors/naturopaths.
But at home, all he had to do was open ChatGPT and spend a few days learning how to prompt it to provide the answers he wanted.
The situations are entirely different.
The status quo being regular doctors? Because that's the status quo, professional doctors.
Having a bot that you can cue and prime towards something is very dangerous in the medical scene.
> Remember, totally legit doctors were peddling opioid pills to patients in the name of "Treating undertreated pain". Profit distorts human minds very easily.
If ChatGPT was trained on those same data that these legit doctors were taught, it would lead to the same diagnosis.
The “MedMD problem” is so classic that a comment next to this one doesn’t have to explain why you always end up with cancer. And the supplement industry pushing PragmaticPulp’s friends’ delusion isn’t exactly discrete either.
All that, while every Western nation has record levels of low physical activity and social isolation: there is a real mismatch between medical advice to civilians and what those tools offer. A convenient, accessible medical dictionary is great, and I have no doubt it can process even more specialized knowledge, given enough strong-arming Epic (the de-facto monopoly on doctor’s note software) and a sprinkle of data engineering. However, that’s not the most apparent issue that this tool has. I’m concerned that OpenAI isn’t more focused on the bigger problem.
Also: with people likely confused about the degree of authority these programs have and their ability to masquerade bullshit as something that is true I wonder if most people would be able to detect whether or not what ChatGPT says is wrong before it harms them.
If they can add a disclaimer they can also say 'Sorry, ChatGPT-4 is not a doctor, consult your doctor'.
This of course would harm those who don't have access to doctors but even there you'd need to prove that it does more good than harm before you'd be allowed to put the output out there. Even the most innocuous medicine has to go through a standardized test before it is released to the public. Without proven efficacy it may well be a net negative.
It's got potential, but someone's gonna have to pony up a lot of money to make it a reality. Probably end up being private equity, and we all know how that ends up
Imagine that after all the base training you re-tune only on medical knowledge letting go all the other languages, programing, history, etc.
I wonder what results we would get.
For sure OpenAI know already.
I want to minimise how many people i see... just make it all automated (and on demand) and that would be great.
https://arxiv.org/abs/2212.13138
What’s your contention exactly? Synopsis mentions it, does the paper not follow up on it?
Basically, the comparison needs to be a lot more comprehensive to be useful.