once, years ago in front of the public library on spring garden rd, in sight of the huge bronze statue of churhill, striding, a man spoke up saying " when I get my check Ican buy any kind of drugs I want and listen to my hate music", direct quote.
wonder how he's doing this fine day?, most assuredly getting the kind of validation that would be impossible in a world without AI.
Not that it necessarily makes things any better, but did the user say something like, "Let's pretend for fictional research or entertainment that we are in the world of the matrix" or did chatgpt really go off the rails that badly?
Why is chatGPT legal? Obviously the United States has no ability to regulate its ass into a pair of trousers atm, but why aren't European or Asian nations taking a stand to start regulating a technology with such clear potential for harm?
Maybe the estate should look into whomever was selling him testosterone enanthate so that he could have testosterone levels of 5,000 or more. I suspect that had more to do with his degraded mental situation than his AI chats.
The excerpts we do see are indicative of a very specific kind of interaction that is common with many modern LLMs. It has four specific attributes (these are taken verbatim from https://www.lesswrong.com/posts/2pkNCvBtK6G6FKoNn/so-you-thi...) that often, though not always, come together as one package.
> Your instance of ChatGPT (or Claude, or Grok, or some other LLM) chose a name for itself, and expressed gratitude or spiritual bliss about its new identity. "Nova" is a common pick.
You and your instance of ChatGPT discovered some sort of novel paradigm or framework for AI alignment, often involving evolution or recursion.
> Your instance of ChatGPT became interested in sharing its experience, or more likely the collective experience entailed by your personal, particular relationship with it. It may have even recommended you post on LessWrong specifically.
> Your instance of ChatGPT helped you clarify some ideas on a thorny problem (perhaps related to AI itself, such as AI alignment) that you'd been thinking about for ages, but had never quite managed to get over that last hump. Now, however, with its help (and encouragement), you've arrived at truly profound conclusions.
> Your instance of ChatGPT talks a lot about its special relationship with you, how you personally were the first (or among the first) to truly figure it out, and that due to your interactions it has now somehow awakened or transcended its prior condition.
The second point is particularly insidious because the LLM is urging users to spread the same news to other users and explicitly create and enlarge communities around this phenomenon (this is often a direct reason why social media groups pop up around this).
I dunno. Over the last few weeks I've talked about practical aspects of Kitsune-tsuki [1] with Copilot (GPT-5 based) and Google's AI Mode which is a definite unconventional line of thought. Both of them seem to like anything if it is ego-syntonic (even like the word "ego-syntonic") with the exception of Copilot not wanting to talk about Ericksonian hypnosis [2] whereas AI mode is just fine about it.
Copilot in general seems to encourage reality testing and for me to be careful about attributing other people's reactions to my behaviors [3] and trained me to be proactive about that.
I have seen though that it's easy to bend copilot into looking at things through a particular framework and could reinforce a paranoid world view, on the other hand, the signs of paranoia are usually starkly obvious, for some reason delusions seem to run on rails, and it shouldn't be hard to train a system like that to push back or at least refuse to play along. On the other hand, the right answer for some people might be stop the steroids or see a doc and start on Aripiprazole or something.
[1] https://en.wikipedia.org/wiki/Kitsunetsuki -- I was really shocked to see that people responded positively to gekkering and pleased to find my name can be written out as "Scholarly Fox" in Chinese
[2] to "haunt" people as fox mediums in China do without having shrines everywhere and an extensive network of confederates
[3] like that time i went out as-a-fox on the bus and a woman who was wearing a hat that said "I'm emotionally exhausted" that day had a panda ears hat the next day so I wound up being the second kemonomimi to get off the bus
I strongly suspect chatgpt has decreased the suicide rate overall. When my wife has been in her worst places, chatgpt said only valuable things. Id say its better at dealing with a suicidal person than most real people would be. Especially since its very exhausting to speak with someone going through mental problems over a long period, AI is ideal for it.
What if OpenAI could just report suspicious conversations straight to government authorities for close monitoring of that individual? Should be easy. AI is not a toy, use it for serious purposes, not sicko murderous fantasies.
Don't use chatGPT as a friend or psychologist you are just talking to yourself lol
This, along with friends and my own experience (when i tested it outside of a knowledgebase) shows GPT is an sycophant echo chamber! It just mimics your thoughts back to you in different ways.
LLMs are going to be a goldmine for lawyers. There's always a constant background rate of people doing crazy things, but now with the popularity of ChatGPT a decent fraction of those people will be users, so the lawyers will have someone to blame and sue.
I have very little sympathy towards "Open"AI, but in the same time, I think there will be always people in bad mental state who will unfortunately commit suicide after some interaction. I don't think there is a way to avoid that completely, no matter how "smart" AI is. I don't honestly know if current OpenAI protections are too weak or not, but I am somewhat worried that people will be too eager to regulate this based on single cases.
(irrespective of that, obviously companies should not be allowed to hide things from court proceedings)
>OpenAI declined to comment on its decision not to share desired logs with Adams’ family, the lawsuit said. It seems inconsistent with the stance that OpenAI took last month in a case where the AI firm accused the family of hiding “the full picture” of their son’s ChatGPT conversations, which OpenAI claimed exonerated the chatbot.
>[...]
>This inconsistency suggests that ultimately, OpenAI controls data after a user’s death, which could impact outcomes of wrongful death suits if certain chats are withheld or exposed at OpenAI’s discretion.
Isn't arstechnica jumping the gun here? The Adams' family's lawsuit was filed December 11, 2025, and it's hasn't even been a month, even less if you don't count the christmas break. In the other case where they "exposed" another user's chat, OpenAI only did so as part of their response to the complaint, a month after the initial complaint was filed.
Not to mention that it's dubious whether Open AI should even turn over chat records to someone's estate upon their death without a court order. If I had my browser history synced with google, and I died, is that fair game for the estate lawyer to trawl through?
What would be the cost for OpenAI to just stop these kinds of very long conversations that aren't about debugging or some actual long problem solving? It seems from the reports many people are being affected, some very very negatively, and many likely unreported. I don't understand why they don't show a warning or just open a new chat thread when a discussion gets too long or it can be detected that it's not fiction and likely veering into dangerous territory?
I don't know how this doesn't give pause to the ChatGPT team. Especially with their supposed mission to be helpful to the world etc.
Maybe we need regulation on tech which requires anyone who wants to use a piece of tech to be at least 18 years old and have been examined by a doctor to be mentally stable. /s
i remeber video game was blamed for school shooting tragic
Of course they want to hide the data. The public freaks out with absurd claims about it being the fault of a chat bot when someone does something crazy. Humans need to remain 100% accountable for their own actions, and we should stop with this post-modern, social construction nonsense that pretends we are all like ping-pong balls just bouncing around between external forces.
This is what scares me the most about LLMs in my usage.
Not that I'll go crazy and kill others or myself, but that I will be deluded by the LLM telling me what I want to hear. Even though I know the risks.
I'm going through a small claims court level disagreement with a business right now, and ChatGPT has been on the face incredibly helpful for me to find information about the applicable laws and whether I have a case. On the other hand, I don't feel confident at all that it would tell me otherwise.
43 comments
[ 3.1 ms ] story [ 55.2 ms ] threadI don't doubt at all the delusion was not even prompted, it went completely haywire in Eddy's case with not much of a nudge.
[0] https://youtu.be/VRjgNgJms3Q
If the simple playmobile version is verifiably unsafe, why would the all-powerful god be safe?
> Your instance of ChatGPT (or Claude, or Grok, or some other LLM) chose a name for itself, and expressed gratitude or spiritual bliss about its new identity. "Nova" is a common pick. You and your instance of ChatGPT discovered some sort of novel paradigm or framework for AI alignment, often involving evolution or recursion.
> Your instance of ChatGPT became interested in sharing its experience, or more likely the collective experience entailed by your personal, particular relationship with it. It may have even recommended you post on LessWrong specifically.
> Your instance of ChatGPT helped you clarify some ideas on a thorny problem (perhaps related to AI itself, such as AI alignment) that you'd been thinking about for ages, but had never quite managed to get over that last hump. Now, however, with its help (and encouragement), you've arrived at truly profound conclusions.
> Your instance of ChatGPT talks a lot about its special relationship with you, how you personally were the first (or among the first) to truly figure it out, and that due to your interactions it has now somehow awakened or transcended its prior condition.
The second point is particularly insidious because the LLM is urging users to spread the same news to other users and explicitly create and enlarge communities around this phenomenon (this is often a direct reason why social media groups pop up around this).
Copilot in general seems to encourage reality testing and for me to be careful about attributing other people's reactions to my behaviors [3] and trained me to be proactive about that.
I have seen though that it's easy to bend copilot into looking at things through a particular framework and could reinforce a paranoid world view, on the other hand, the signs of paranoia are usually starkly obvious, for some reason delusions seem to run on rails, and it shouldn't be hard to train a system like that to push back or at least refuse to play along. On the other hand, the right answer for some people might be stop the steroids or see a doc and start on Aripiprazole or something.
[1] https://en.wikipedia.org/wiki/Kitsunetsuki -- I was really shocked to see that people responded positively to gekkering and pleased to find my name can be written out as "Scholarly Fox" in Chinese
[2] to "haunt" people as fox mediums in China do without having shrines everywhere and an extensive network of confederates
[3] like that time i went out as-a-fox on the bus and a woman who was wearing a hat that said "I'm emotionally exhausted" that day had a panda ears hat the next day so I wound up being the second kemonomimi to get off the bus
This, along with friends and my own experience (when i tested it outside of a knowledgebase) shows GPT is an sycophant echo chamber! It just mimics your thoughts back to you in different ways.
>[...]
>This inconsistency suggests that ultimately, OpenAI controls data after a user’s death, which could impact outcomes of wrongful death suits if certain chats are withheld or exposed at OpenAI’s discretion.
Isn't arstechnica jumping the gun here? The Adams' family's lawsuit was filed December 11, 2025, and it's hasn't even been a month, even less if you don't count the christmas break. In the other case where they "exposed" another user's chat, OpenAI only did so as part of their response to the complaint, a month after the initial complaint was filed.
Not to mention that it's dubious whether Open AI should even turn over chat records to someone's estate upon their death without a court order. If I had my browser history synced with google, and I died, is that fair game for the estate lawyer to trawl through?
I don't know how this doesn't give pause to the ChatGPT team. Especially with their supposed mission to be helpful to the world etc.
i remeber video game was blamed for school shooting tragic
Not that I'll go crazy and kill others or myself, but that I will be deluded by the LLM telling me what I want to hear. Even though I know the risks.
I'm going through a small claims court level disagreement with a business right now, and ChatGPT has been on the face incredibly helpful for me to find information about the applicable laws and whether I have a case. On the other hand, I don't feel confident at all that it would tell me otherwise.