All regulations involving AI just make me wonder how the hell anyone can draw a line between statistics and whatever ""AI"" means in 2024. Getting angry at the moment every time i see "AI" because the terminology is just nonsense to me. Is it just neural networks now?
"Statistics" = models with interpretable decision outcomes
"AI" = models without
It's just a little bit easier to hold someone accountable for a logistic regression with predictive bias than for a neural network with similar issues.
I believe it’s possible with decision trees and forests, even though they’re very complex inside, to calculate the contribution that each independent variable made to the final result. Of course, I suppose those haven’t been deemed “AI” for a few years.
Sure, you can step through it and see what's occurring, but there's zero explanation to accompany it. To fully reverse engineer and explain a reasonably complex AI model would be quite the undertaking. Instead, we have prompt engineering. It's easier to figure out how to ask the black box nicely/properly to do what you want than to open it up and optimize/tune it proper. If we assume that we get that far, I fully expect the resulting explanation of the black box internals wouldn't be considered a reasonable/reliable basis for making medical decisions.
As long as the model is deterministic in operation, even if the process of building the model is non-deterministic, can't decisions be interpreted by rerunning the model with tweaked inputs to find what would be needed to change the outcome?
For example suppose a bank rejects a credit application and they need to tell the applicant why. Repeatedly add $1000 to the income input and check again until until the model approves, and then tell the application they were rejected for insufficient income and how much more they would need to pass.
Yes, this is a counterfactual line of reasoning. It's certainly legit, and I believe it's an active field of research in ML, but at the end of the day, you're still poking around the inputs of a black-box model as opposed to examining a white-box model.
Instead, a physician working for the insurance company will use some sort of GPT to come up with an excuse to deny care and then they will write that up in their own words to deny care.
as a med student, those guys are an embarrassment to us all. I had to deal with those guys to get my insurance to cover recombinant antibodies for the treatment of several autoimmune diseases. They haven’t practiced clinically for a decade, haven’t kept up with research ever since they left residency, but still don’t see any problem with denying anything but the cheapest immunosuppressant, no matter how bad the side effects may be. Screw them all, i wish them nothing but the worst
That is a weird place to focus the blame when the entire healthcare system is equally terrible or worse, and is clearly the result of plain capitalism.
The physician who gives legitimacy to denying coverage at an insurance company is at the very heart of what you're complaining about. As the other poster has pointed out, and many physicians online, they are so out of practice and out of touch they basically say whatever the insurance company wants them to for money.
Medical ethics really should preclude what these insurance company physicians are doing, which is sometimes rubber-stamping 5000 denials a day. Maybe an organized campaign of board complaints is in order.
Propublica has done an excellent slate of investigatory articles recently about how insurance denials actually work.
maybe there are some differences from medical system to medical system, but that’s decidedly not what those characters are over here in Germany. This isn’t a case of student loans going overboard, we have it pretty good in this regard, the physicians working as insurance consultants/experts are almost exclusively retired physicians with decades of clinical practice long behind them. At least over here, if you don’t screw up BAD, you’ll live a pretty cushy life as long as you stick to your means.
There’s a difference between doing an unsavory job to the best of one’s abilities and calling women’s mental health issues "hysterics" and invoking psychosomatics to avoid paying out after someone was badly injured horseback riding and are bedridden due to debilitating neurological disorders following her injuries.
There’s a decidedly difference between being strict, maybe even excessively so, whilst doing a controversial job and systematically being misleading, manipulative and untruthful when confronted with established standards of care.
I fully get having to yield to the way the world works, and i sincerely feel for your view. My parents have basically disowned me and trying to sustain myself through medical school is taking a fat toll on me.
But I’m sorry, but "savying" away any sort of criticism of the conduct of our profession on the basis of "that’s not of the real world works" has it’s limits. Maybe you haven’t been at the receiving end of this sort of ordeal, but i can attest you first hand that this group of physicians, at least here, has issues way beyond having to sustain themselves. Maybe our generation will be different, but the ease of your dismissal is kind of troubling to me.
A few months ago i came across an interview with a physician who worked for one of the online pill mills that sprung up after the US loosened the rules for RXing scheduled substance via telemedicine. He pretty much approved Benzos and Adderall all day. He invoked pretty much the very same argument you have, and I’m sure his situation was everything but easy. The pressures of becoming a US physician are very alien to me, so maybe i just don’t get it. But "well, that’s how the world keeps spinning, grow up kiddo" isn’t a carte blanche for being oblivious to the damage you’re doing. You have sworn or will swear an oath. Being a mercenary liar, misrepresenting medicine for monetary gain is not quite what was implied there. If you don’t see a glaring issue with behaving this way, maybe you should look into an alternative career path.
>ease of your dismissal is kind of troubling to me.
I'm sorry you perceived my comment this way. It's not a dismissal, rather a charge to maintain your purity.
>The pressures of becoming a US physician are very alien to me, so maybe i just don’t get it.
The US training schedules for physicians are ridiculous, absolutely. I never really "got it," either, which is why I do not practice medicine.
>"well, that’s how the world keeps spinning, grow up kiddo" isn’t a carte blanche for being oblivious to the damage you’re doing
Please consider how this misdirected ANGER resonated with myself, others.
>If you don’t see a glaring issue with behaving this way, maybe you should look into an alternative career path.
I did, having left medical school over a decade ago [I wouldn't have been able to maintain the purity I saw in your above-comment, so I changed professions].
----
Best of luck in your studies. I truly hope one day you'll "make it" and be able to help people in the real world.
I think that’s ok from the perspective of the discussion at hand because we are still talking about an actual human’s terrible decision having their name attached to it. I think the system used to deny care being comically broken is a wholly different issue.
The AMA is one of the primary reasons US healthcare is SO FUCKED [by design].
As the first of many examples, limiting accredited residency program seats nationwide places an artificial restriction on produced-physicians.
I'm very happy to see increase in ancillary licensure/practice, although I know this just increases "the race to the bottom" which echoes so familiarly in regulatory capture / oligarchy.
I don’t see how we can use ai and not have it generate new biases.
I also feel like biases have been baked into insurance data in a discriminatory way for a long time — think about how much car insurance is for a 16 year old boy.
Yes but consider the alternative - how will we eliminate bias without it?
Having worked in medical billing, it is very scary how medicine as an applied science is carrying around a huge ball and chain of “insurance hasn’t figured out that their medical model is decades out of date.”
Big, complex, data intensify problems are not going to be solved by an army of humans on legacy ERPs.
Everyone pays the same regardless of age, sex, etc? I understand they things are the way they are but part of change is making difficult decisions and tradeoffs.
I cannot even describe how many problems are caused by medicine being shackled to the whims of insurers in terms of how stultifying it is and how many bad incentives it creates to workaround the limitations of insurance.
> In an example involving a decision to terminate post-acute care services, an algorithm or software tool can be used to assist providers or MA plans in predicting a potential length of stay, but that prediction alone cannot be used as the basis to terminate post-acute care services.
So, the Random Forest can say, "No coverage", just so long as the human hits "Accept computer recommendations"
How about just prohibiting them from denying coverage at all, except for bona fide fraud? So much of the rot of the healthcare system stems from this broken regulatory approach of thinking that corporate bureaucrats will somehow "manage" care better than patients and doctors can. These "insurance" companies should be relegated to a purely financial role, as a basic table stakes reform.
There’s no reason to get a MRI for a basic headache, but if it’s free, why not?
And attempts to solve this just muddle it worse, but some could help - nobody is going to voluntarily take insulin, for example, so just make those impossible to deny.
Yes, obviously you should "argue" and require an explanation of the doctor's reasoning. I just literally rejected doing an MRI this past weekend, in an emergency context, that would have been financially "free". The things it might have found were already disimplied by bloodwork, or were rare with no treatment options, and in my opinion the amount of iatrogenic harm from the procedure would have been unjustified.
A patient or trusted advocated needs to be involved in a patient's own care, and the current "insurance" system does a great job at destroying this as doctors focus on "insurance" companies and associated bureaucracy as decision makers rather than patients. IMO this is precisely where the request to do an MRI was coming from in the first place - bureaucratic ass-covering.
Placing an IV after they had already botched it three times, living on lactated Ringer's instead of real food for the better part of a day, IV sedation anesthesia and its possible complications, (I didn't investigate side effects of the contrast), and the further destruction of bodily autonomy and resulting lack of alertness that comes from all those.
Yikes. Seems like a pretty serious MRI. Good luck.
On one hand, this defensive practice of medicine is objectionable. On the other, it only exists because people like to sue providers when outcomes are bad. I’m glad you had a good outcome.
Everything turned out fine, thank you. I do get why in an emergency context doctors want to do as many tests as possible in parallel, and have very little context to guide them otherwise. But also I don't have much sympathy for things "only" being a certain way because of some other thing. The medical industry is a whole set of self-reinforcing terrible dynamics, which is why we need sweeping reform.
I've also got to wonder how much this narrative bemoaning MRIs for "headaches" is because doctors have rightfully moved to doing less harmful MRIs instead of less expensive X-ray CTs.
Standard "insurance" generally includes copayments for services, sometimes even percentage based. Insurance companies would obviously adapt to such a reform, and my entire point is to align the incentives better. Over the long term, "insurance" companies operating on a cost-plus basis don't even really want to keep costs down, rather denials stem from needing to make sure any payouts are bureaucratically justifiable.
Maybe in some cases, though overall cost of living in Australia is slightly higher than the US while still spending about half what we do on medical costs.
Whew, I was afraid this AI thing was going to mean we were all out of a job. But machines cannot replace the human ability to take the blame. You need a warm body for that.
54 comments
[ 2.6 ms ] story [ 131 ms ] thread"AI" = models without
It's just a little bit easier to hold someone accountable for a logistic regression with predictive bias than for a neural network with similar issues.
Decisions based on statistical models can be fully audited and understood, and the results are reproducible because the algorithm is well documented.
The same can't be said of AI models.
For example suppose a bank rejects a credit application and they need to tell the applicant why. Repeatedly add $1000 to the income input and check again until until the model approves, and then tell the application they were rejected for insufficient income and how much more they would need to pass.
Propublica has done an excellent slate of investigatory articles recently about how insurance denials actually work.
Oftentimes, the "embarrassment to us all" is just trying to pay his own bills/obligations [not saying this is "correct" — it's not].
There’s a difference between doing an unsavory job to the best of one’s abilities and calling women’s mental health issues "hysterics" and invoking psychosomatics to avoid paying out after someone was badly injured horseback riding and are bedridden due to debilitating neurological disorders following her injuries.
There’s a decidedly difference between being strict, maybe even excessively so, whilst doing a controversial job and systematically being misleading, manipulative and untruthful when confronted with established standards of care.
I fully get having to yield to the way the world works, and i sincerely feel for your view. My parents have basically disowned me and trying to sustain myself through medical school is taking a fat toll on me.
But I’m sorry, but "savying" away any sort of criticism of the conduct of our profession on the basis of "that’s not of the real world works" has it’s limits. Maybe you haven’t been at the receiving end of this sort of ordeal, but i can attest you first hand that this group of physicians, at least here, has issues way beyond having to sustain themselves. Maybe our generation will be different, but the ease of your dismissal is kind of troubling to me.
A few months ago i came across an interview with a physician who worked for one of the online pill mills that sprung up after the US loosened the rules for RXing scheduled substance via telemedicine. He pretty much approved Benzos and Adderall all day. He invoked pretty much the very same argument you have, and I’m sure his situation was everything but easy. The pressures of becoming a US physician are very alien to me, so maybe i just don’t get it. But "well, that’s how the world keeps spinning, grow up kiddo" isn’t a carte blanche for being oblivious to the damage you’re doing. You have sworn or will swear an oath. Being a mercenary liar, misrepresenting medicine for monetary gain is not quite what was implied there. If you don’t see a glaring issue with behaving this way, maybe you should look into an alternative career path.
I'm sorry you perceived my comment this way. It's not a dismissal, rather a charge to maintain your purity.
>The pressures of becoming a US physician are very alien to me, so maybe i just don’t get it.
The US training schedules for physicians are ridiculous, absolutely. I never really "got it," either, which is why I do not practice medicine.
>"well, that’s how the world keeps spinning, grow up kiddo" isn’t a carte blanche for being oblivious to the damage you’re doing
Please consider how this misdirected ANGER resonated with myself, others.
>If you don’t see a glaring issue with behaving this way, maybe you should look into an alternative career path.
I did, having left medical school over a decade ago [I wouldn't have been able to maintain the purity I saw in your above-comment, so I changed professions].
----
Best of luck in your studies. I truly hope one day you'll "make it" and be able to help people in the real world.
Why doesn't the AMA expel these doctors from its ranks?
As the first of many examples, limiting accredited residency program seats nationwide places an artificial restriction on produced-physicians.
I'm very happy to see increase in ancillary licensure/practice, although I know this just increases "the race to the bottom" which echoes so familiarly in regulatory capture / oligarchy.
I also feel like biases have been baked into insurance data in a discriminatory way for a long time — think about how much car insurance is for a 16 year old boy.
Having worked in medical billing, it is very scary how medicine as an applied science is carrying around a huge ball and chain of “insurance hasn’t figured out that their medical model is decades out of date.”
Big, complex, data intensify problems are not going to be solved by an army of humans on legacy ERPs.
No idea how to fix the problem though.
Start an insurance company
It sounds impossible, but every one of these giants were founded in the past century.
They can be beaten with enough capital, power and ingenuity.
But that’s not gonna be done by a pre-seed startup with $1M in the bank and some cool products.
Genuine change requires a long view and a willingness to compete against a cartel.
So, the Random Forest can say, "No coverage", just so long as the human hits "Accept computer recommendations"
There’s no reason to get a MRI for a basic headache, but if it’s free, why not?
And attempts to solve this just muddle it worse, but some could help - nobody is going to voluntarily take insulin, for example, so just make those impossible to deny.
Because an MRI is an exhausting, irritating experience.
My understanding is that we’re already “over testing”, though would defer to someone with a citation on the topic.
A patient or trusted advocated needs to be involved in a patient's own care, and the current "insurance" system does a great job at destroying this as doctors focus on "insurance" companies and associated bureaucracy as decision makers rather than patients. IMO this is precisely where the request to do an MRI was coming from in the first place - bureaucratic ass-covering.
On one hand, this defensive practice of medicine is objectionable. On the other, it only exists because people like to sue providers when outcomes are bad. I’m glad you had a good outcome.
I've also got to wonder how much this narrative bemoaning MRIs for "headaches" is because doctors have rightfully moved to doing less harmful MRIs instead of less expensive X-ray CTs.
As of now, medical care in the US is about 2x-3x the cost it is in other developed countries.
Yes, it sounds naive, but it is true.