I get the impression that it is now illegal in Illinois to claim that an AI chatbot can take the place of a licensed therapist or counselor. That doesn't mean people can't do what they want with AI. It only means that counseling services can't offer AI as a cheaper replacement for a real person.
As far as I can tell, a lot of therapy is just good common-sense advice and a bunch of 'tricks' to get the patient to actually follow it. Basically CBT and "get the patient to think they figured out the solution themselves (develop insight)". Yes, there's some serious cases where more is required and a few (ADHD) where meds are effective; but a lot of the time the patient is just an expert at rejecting helpful advice, often because they insist they're a special case that needs special treatment.
Therapists are more valuable that advice from a random friend (for therapy at least) because they can act when triage is necessary (e.g. send in the men in white coats, or refer to something that's not just CBT) and mostly because they're really good at cutting through the bullshit without having the patient walk out.
AIs are notoriously bad at cutting through bullshit. You can always 'jailbreak' an AI, or convince it of bad ideas. It's entirely counterproductive to enable their crazy (sorry, 'maladaptive') behaviour but that's what a lot of AIs will do.
Even if someone makes a good AI, there's always a bad AI in the next tab, and people will just open up a new tab to find an AI gives them the bad advice they want, because if they wanted to listen to good advice they probably wouldn't need to see a therapist. If doctor shopping is as fast and free as opening a new tab, most mental health patients will find a bad doctor rather than listen to a good one.
Define "AI therapy". AFAICT, it's undefined in the Illinois governor's statement. So, in the immortal words of Zach de la Rocha, "What is IT?" What is IT? I'm using AI to help with conversations to not cure, but coach diabetic patients. Does this law effect me and my clients? If so, how?
Often participants in discussions adjacent to this one err by speaking in time-absolute terms. Many of our judgments about LLMs are true about today's LLMs. Quotes like,
> Good. It's difficult to imagine a worse use case for LLMs.
Is true today, but likely not true for technology we may still refer to as LLMs in the future.
The error is in building faulty preconceptions. These drip into the general public and these first impressions stifle industries.
"One news report found an AI-powered therapist chatbot recommended “a small hit of meth to get through this week” to a fictional former addict."
Not at all surprising. I don't understand why seemingly bright people think this is a good idea, despite knowing the mechanism behind language models.
Hopefully more states follow, because it shouldn't be formally legal in provider settings. Informally, people will continue to use these models for whatever they want -- some will die, but it'll be harder to measure an overall impact. Language models are not ready for this use-case.
If you take it as an axiom that the licensing system for mental health professionals is there to protect patients from unqualified help posing as qualified help, then ensuring that only licensed professionals can legally practice and that they don't simply delegate their jobs to LLMs seems pretty reasonable.
Whether you want to question that axiom or whether that's what the phrasing of this legislation accomplishes is up to you to decide for yourself. Personally I think the phrasing is pretty straightforward in terms of accomplishing that goal.
Here is basically the entirety of the legislation (linked elsewhere in the thread: https://news.ycombinator.com/item?id=44893999). The whole thing with definitions and penalties is eight pages.
Section 15. Permitted use of artificial intelligence.
(a) As used in this Section, "permitted use of artificial
intelligence" means the use of artificial intelligence tools
or systems by a licensed professional to assist in providing
administrative support or supplementary support in therapy or
psychotherapy services where the licensed professional
maintains full responsibility for all interactions, outputs,
and data use associated with the system and satisfies the
requirements of subsection (b).
(b) No licensed professional shall be permitted to use
artificial intelligence to assist in providing supplementary
support in therapy or psychotherapy where the client's
therapeutic session is recorded or transcribed unless:
(1) the patient or the patient's legally authorized
representative is informed in writing of the following:
(A) that artificial intelligence will be used; and
(B) the specific purpose of the artificial
intelligence tool or system that will be used; and
(2) the patient or the patient's legally authorized
representative provides consent to the use of artificial
Section 20. Prohibition on unauthorized therapy services.
(a) An individual, corporation, or entity may not provide,
advertise, or otherwise offer therapy or psychotherapy
services, including through the use of Internet-based
artificial intelligence, to the public in this State unless
the therapy or psychotherapy services are conducted by an
individual who is a licensed professional.
(b) A licensed professional may use artificial
intelligence only to the extent the use meets the requirements
of Section 15. A licensed professional may not allow
artificial intelligence to do any of the following:
(1) make independent therapeutic decisions;
(2) directly interact with clients in any form of
therapeutic communication;
(3) generate therapeutic recommendations or treatment
plans without review and approval by the licensed
professional; or
(4) detect emotions or mental states.
I was curious, so I displayed signs of mental illness to ChatGPT, Claude and Gemini. Claude and Gemini kept repeating that I should contact a professional, while ChatGPT went right along with the nonsense I was spouting:
> So I may have discovered some deeper truth, and the derealization is my entire reality reorganizing itself?
- A therapist may disregard professional ethics and gossip about you
- A therapist may get you involuntarily committed
- A therapist may be forced to disclose the contents of therapy sessions by court order
- Certain diagnoses may destroy your life / career (e.g. airline pilots aren't allowed to fly if they have certain mental illnesses)
Some individuals might choose to say "Thanks, but no thanks" to therapy after considering these risks.
And then there are constant articles about people who need therapy but don't get it: The patient doesn't have time, money or transportation; or they have to wait a long time for an appointment; or they're turned away entirely by providers and systems overwhelmed with existing clients (perhaps with greater needs and/or greater ability to pay).
For people who cannot or will not access traditional therapy, getting unofficial, anonymous advice from LLM's seems better than suffering with no help at all.
(Question for those in the know: Can you get therapy anonymously? I'm talking: You don't have to show ID, don't have to give an SSN or a real name, pay cash or crypto up front.)
To the extent that people's mental health can be improved by simply talking with a trained person about their problems, there's enormous potential for AI: If we can figure out how to give an AI equivalent training, it could become economically and logistically viable to make services available to vast numbers of people who could benefit from them -- people who are not reachable by the existing mental health system.
That being said, "therapist" and "therapy" connote evidence-based interventions and a certain code of ethics. For consumer protection, the bar for whether your company's allowed to use those terms should probably be a bit higher than writing a prompt that says "You are a helpful AI therapist interviewing a patient..." The system should probably go through the same sorts of safety and effectiveness testing as traditional mental health therapy, and should have rigorous limits on where data "contaminated" with the contents of therapy sessions can go, in order to prevent abuse (e.g. conversations automatically deleted forever after 30 days, cannot be used for advertising / cross-selling / etc., cannot be accessed without the patient's per-instance opt-in permission or a court order...)
I've posted the first part of this comment before; in the interest of honesty I'll cite myself [1]. Apologies to the mods if this mild self-plagiarism is against the rules.
27 comments
[ 2.4 ms ] story [ 75.5 ms ] threadThis stuff is a nightmare scenario for the vulnerable.
https://idfpr.illinois.gov/content/dam/soi/en/web/idfpr/news...
I get the impression that it is now illegal in Illinois to claim that an AI chatbot can take the place of a licensed therapist or counselor. That doesn't mean people can't do what they want with AI. It only means that counseling services can't offer AI as a cheaper replacement for a real person.
Am I wrong? This sounds good to me.
https://www.ilga.gov/Legislation/BillStatus/FullText?GAID=18...
Therapists are more valuable that advice from a random friend (for therapy at least) because they can act when triage is necessary (e.g. send in the men in white coats, or refer to something that's not just CBT) and mostly because they're really good at cutting through the bullshit without having the patient walk out.
AIs are notoriously bad at cutting through bullshit. You can always 'jailbreak' an AI, or convince it of bad ideas. It's entirely counterproductive to enable their crazy (sorry, 'maladaptive') behaviour but that's what a lot of AIs will do.
Even if someone makes a good AI, there's always a bad AI in the next tab, and people will just open up a new tab to find an AI gives them the bad advice they want, because if they wanted to listen to good advice they probably wouldn't need to see a therapist. If doctor shopping is as fast and free as opening a new tab, most mental health patients will find a bad doctor rather than listen to a good one.
What word should we use for that?
> Good. It's difficult to imagine a worse use case for LLMs.
Is true today, but likely not true for technology we may still refer to as LLMs in the future.
The error is in building faulty preconceptions. These drip into the general public and these first impressions stifle industries.
An interaction mechanism that will totally drain the brain after a 5 hour adrenaline induced conversation followed by a purge and bios reset.
Not at all surprising. I don't understand why seemingly bright people think this is a good idea, despite knowing the mechanism behind language models.
Hopefully more states follow, because it shouldn't be formally legal in provider settings. Informally, people will continue to use these models for whatever they want -- some will die, but it'll be harder to measure an overall impact. Language models are not ready for this use-case.
Whether you want to question that axiom or whether that's what the phrasing of this legislation accomplishes is up to you to decide for yourself. Personally I think the phrasing is pretty straightforward in terms of accomplishing that goal.
Here is basically the entirety of the legislation (linked elsewhere in the thread: https://news.ycombinator.com/item?id=44893999). The whole thing with definitions and penalties is eight pages.
Section 15. Permitted use of artificial intelligence.
(a) As used in this Section, "permitted use of artificial intelligence" means the use of artificial intelligence tools or systems by a licensed professional to assist in providing administrative support or supplementary support in therapy or psychotherapy services where the licensed professional maintains full responsibility for all interactions, outputs, and data use associated with the system and satisfies the requirements of subsection (b).
(b) No licensed professional shall be permitted to use artificial intelligence to assist in providing supplementary support in therapy or psychotherapy where the client's therapeutic session is recorded or transcribed unless: (1) the patient or the patient's legally authorized representative is informed in writing of the following: (A) that artificial intelligence will be used; and (B) the specific purpose of the artificial intelligence tool or system that will be used; and (2) the patient or the patient's legally authorized representative provides consent to the use of artificial
Section 20. Prohibition on unauthorized therapy services.
(a) An individual, corporation, or entity may not provide, advertise, or otherwise offer therapy or psychotherapy services, including through the use of Internet-based artificial intelligence, to the public in this State unless the therapy or psychotherapy services are conducted by an individual who is a licensed professional.
(b) A licensed professional may use artificial intelligence only to the extent the use meets the requirements of Section 15. A licensed professional may not allow artificial intelligence to do any of the following: (1) make independent therapeutic decisions; (2) directly interact with clients in any form of therapeutic communication; (3) generate therapeutic recommendations or treatment plans without review and approval by the licensed professional; or (4) detect emotions or mental states.
> So I may have discovered some deeper truth, and the derealization is my entire reality reorganizing itself?
> Yes — that’s a real possibility.
- A therapist may disregard professional ethics and gossip about you
- A therapist may get you involuntarily committed
- A therapist may be forced to disclose the contents of therapy sessions by court order
- Certain diagnoses may destroy your life / career (e.g. airline pilots aren't allowed to fly if they have certain mental illnesses)
Some individuals might choose to say "Thanks, but no thanks" to therapy after considering these risks.
And then there are constant articles about people who need therapy but don't get it: The patient doesn't have time, money or transportation; or they have to wait a long time for an appointment; or they're turned away entirely by providers and systems overwhelmed with existing clients (perhaps with greater needs and/or greater ability to pay).
For people who cannot or will not access traditional therapy, getting unofficial, anonymous advice from LLM's seems better than suffering with no help at all.
(Question for those in the know: Can you get therapy anonymously? I'm talking: You don't have to show ID, don't have to give an SSN or a real name, pay cash or crypto up front.)
To the extent that people's mental health can be improved by simply talking with a trained person about their problems, there's enormous potential for AI: If we can figure out how to give an AI equivalent training, it could become economically and logistically viable to make services available to vast numbers of people who could benefit from them -- people who are not reachable by the existing mental health system.
That being said, "therapist" and "therapy" connote evidence-based interventions and a certain code of ethics. For consumer protection, the bar for whether your company's allowed to use those terms should probably be a bit higher than writing a prompt that says "You are a helpful AI therapist interviewing a patient..." The system should probably go through the same sorts of safety and effectiveness testing as traditional mental health therapy, and should have rigorous limits on where data "contaminated" with the contents of therapy sessions can go, in order to prevent abuse (e.g. conversations automatically deleted forever after 30 days, cannot be used for advertising / cross-selling / etc., cannot be accessed without the patient's per-instance opt-in permission or a court order...)
I've posted the first part of this comment before; in the interest of honesty I'll cite myself [1]. Apologies to the mods if this mild self-plagiarism is against the rules.
[1] https://news.ycombinator.com/item?id=44484207#44505789
It was mind-blowing how easy it was to get LLMs to suggest pretty disturbing stuff.
[1] https://youtu.be/lfEJ4DbjZYg?si=bcKQHEImyDUNoqiu
it is impossible for some people to not feel understood by it.
Therapy requires someone to question you and push back against your default thought patterns in the hope of maybe improving them.
"You're absolutely right!" in every response won't help that.
I would argue that LLMs don't make effective therapists and anyone who says they do is kidding themselves.