It's not surprising to you or me, but think of someone less technologically proficient. They won't know the difference, and if the AI complies with their request in a convincing way, they may very well believe it.
Yeah, there was a paper about AI excellence in diagnosing radiology images a few (5-10 now?) years ago, that turned out to be largely driven by accuracy skewed bias in the input data (IIRC it was using the existence of lab-style lighting, rulers, etc as its diagnosis).
Chat-GPT and all the LLMs are doing essentially the same "predict the next work in a sentence" to sound as plausibly real as possible, which is essentially the same gimmick, only we call it hallucination due to VC funded AI firms need to pretend LLMs understand anything.
On some days, Claude (and other LLMs) will hallucinate on one input and produce garbage and in the next update it will then believe something else in the next unpredictably.
Perhaps using general purpose LLMs for safety critical use cases without humans isn't a good idea. This is why we have a 'human in the loop' giving the feedback to resolve the trust issue that LLMs lack.
Maybe we should trust LLMs like Claude to pilot a plane without any human pilots on board and raise VC money on the idea on replacing all pilots since we keep having these conversations on LLMs being compared with humans and countless VCs screaming over robots replacing humans in all occupations and 'accelerating everything' which could not be more wrong.
Bloody hell, this sort of thing is going to get people killed. The magic box is not a person, let alone a qualified doctor. Do not ask the magic box questions about anything important.
My girlfriend has had some medical issues for a few years and none of the 3-4 doctors she has seen have been able to diagnose her. That is not to say that no doctor can. Just not the ones she has seen.
I do not believe in the infallibility of medical professionals. And I have seen AI totally get it wrong every day. But, I am of the opinion that we are not very far from the point where AI will be better at diagnosing issues than an average doctor. That isn't a high bar. An average doctor isn't very good.
I think AI is almost there already for an average programmer. An average programmer isn't very good either.
17 comments
[ 3.6 ms ] story [ 54.1 ms ] thread2) I am not sure its surprising that a general purpose model is not great at reading MRI scans.
ChatGPT does specifically disallow medical advice and generally it is RLHFed not to provide it: https://openai.com/policies/usage-policies
Chat-GPT and all the LLMs are doing essentially the same "predict the next work in a sentence" to sound as plausibly real as possible, which is essentially the same gimmick, only we call it hallucination due to VC funded AI firms need to pretend LLMs understand anything.
G.Hinton, 2016
> The error rate produced by a deep learning based methods dropped below that achieved by human observers in 2015 for the first time
Essentially we just need good implementations at this point. We're there, just not widely deployed.
Perhaps using general purpose LLMs for safety critical use cases without humans isn't a good idea. This is why we have a 'human in the loop' giving the feedback to resolve the trust issue that LLMs lack.
Maybe we should trust LLMs like Claude to pilot a plane without any human pilots on board and raise VC money on the idea on replacing all pilots since we keep having these conversations on LLMs being compared with humans and countless VCs screaming over robots replacing humans in all occupations and 'accelerating everything' which could not be more wrong.
I do not believe in the infallibility of medical professionals. And I have seen AI totally get it wrong every day. But, I am of the opinion that we are not very far from the point where AI will be better at diagnosing issues than an average doctor. That isn't a high bar. An average doctor isn't very good.
I think AI is almost there already for an average programmer. An average programmer isn't very good either.