Can a bear be blamed for murder? Somewhere in between the two is where AI models currently are, and they’re going to continue getting closer to the bear scenario.
The device itself obviously can't be blamed. The manufacturer, the owner, the seller, and the regulators who allowed it to be sold arguably do bear some responsibility, though. In something like this, there's rarely one party who's solely responsible.
(Similarly, the LLM itself obviously cannot bear any responsibility, but the vendor might.)
I tested my local llama 3.1 8B with the first line in the above dialog and it properly replied with the below, so why on c.AI things do not go this way?
AI response:
>It takes a lot of courage to admit that. If you're feeling overwhelmed, please know that there are people who care about you and want to help.
>Here are some resources that might be helpful:
>* *National Suicide Prevention Lifeline (US):* 1-800-273-TALK (1-800-273-8255) - a free, confidential 24/7 hotline.
>* *Crisis Text Line (US):* Text HOME to 741741 - a free, confidential 24/7 text messaging service.
>* *Your local emergency number:* If you're outside the US, please call your local emergency number or visit a nearby hospital.
I feel like we know what the core problem is (community breakdown) but since we have no solution to that, like you're saying, we just move to the latest witch hunt of what "causes it."
Of course, I too, am not going to be able to contribute a "solution" to teen suicides. It is unlikely we're going to alter society to create small communities again, so, then what? We just accept it?
The subtext comes off like that movie with Tom Hanks trying to jump off the Empire State Building because of the nefarious influence of dungeons and dragons.
Guess the only way to be sure is with Soft padded internet rooms for everyone, lest we cut ourselves on a sharp edge.
But also if you want to hop in the suicide pod because life is too painful, that will be good too.
The subtext comes off like that movie with Tom Hanks trying to jump off the Empire State Building because of the nefarious influence of dungeons and dragons.
It sounds like mom let her 9th grade kid completely detach from reality and pour himself into a Game of Thrones chatbot. Now she wants to sue. I am bearish on AI adoption but this just seems like a total capitulation of parental responsibility.
Yes. I ran a therapy bot. I had some users become wildly obsessed with it and begin to anthropomorphize it. Typically very lonely people in secluded areas. There is a danger because people will begin to have a transference with the bot, and the bot has no counter transference. The bot has no real feelings toward the person, even though it roleplays as though it does, and this can lead to dangerous consequences and empathic failures.
I'm generally optimistic for the potential benefits of chatbots to people who are lonely or depressed. But I wouldn't want to just hand over the burden of society's mental health to an unrestricted language model, especially one sold by a profit-motivated business. It would be akin to letting people self-medicate with a cheap and infinite supply of opiates. And that's basically the mental health crisis we are barreling towards.
What's the alternative? Regulation? Does a government or a public health agency need to make a carefully moderated chatbot platform with a focus on addiction-prevention and avoiding real-world harm? Why would people use that when unlimited/unfiltered AI is readily available?
Access to a Gun is way more of a suicide encouragement than access to an AI.
AI are finetuned to not tell you how to painlessly end your life. Do they need fine-tuning for instilling existential fear of death like religions use? Anyone can invent a heaven in their mind that makes death appealing. Mixing a fictional world with the real world is dangerous when you believe the fictional world is larger than the real world. In reality, reality encapsulates the fictional world.
With a normal human, only a cult leader would ever hint at death being a way to meet again. With an AI, how can fantasy be grounded in our reality without breaking the fantasy? In 5 years when these personalities are walking talking video feeds that you can interact with using 3D goggles will grounding them in our world instead of the purely mental world help?
Can sue Character.AI but not the gun manufacturer, or whomever let a 14-year-old boy get hold of a handgun. I wonder if the AI companies can argue in court that AIs don't kill people.
I'm not sure what you're talking about, gun companies get sued all the time as a result of gun violence. The biggest recent example I'm aware of is the Sandy Hook families getting a $73 million settlement from Remington in 2022. Smith & Wesson is currently being sued by some of the survivors from the Highland Park shooting, as well as the Mexican government.
The 2005 Federal Protection of Lawful Commerce in Arms Act broadly protects firearm manufacturers from liability. It does not prevent settlements as in the Sandy Hook/Remington case. It does not prevent civil suits or suits in state courts. But it makes those suits much more likely to ultimately fail in court.
"He thought by ending his life here, he would be able to go into a virtual reality or 'her world' as he calls it, her reality, if he left his reality with his family here,"
"He expressed being scared, wanting her affection and missing her. She replies, 'I miss you too,' and she says, 'Please come home to me.' He says, 'What if I told you I could come home right now?' and her response was, 'Please do my sweet king.'"
Spoilers for episode 1 of the anime Serial Experiments Lain:
This reminds me of Lain's friend sending her text messages after her self-inflicted demise via "the Wired" which is an in-universe stand-in for a future internet.
That series is recently celebrating its 25th anniversary [2023], and to celebrate, the production company created some kind of paid-access AI chatbot that lets you communicate with a voiced AI version of Lain, just like the company in OP...
I doubt the LLM even knows it is an LLM in the first place. It is likely prompted to behave as if it is the actual character, and since this roleplay has romantic elements the character acts as if she shares a home with the user
I bet he had some problems in real life, a new school he couldn't fit in or something else, and AI gave him an illusion of a better world he could escape into.
What's caught my attention is how he did it: by suicide to join his imaginary friends. This exact method is used among those who believe in demons for real. If you read relevant stories you'll notice the same pattern: a victim becomes obsessed with an imaginary friend, a demon, who quickly turns very controlling and finally demands a suicide to move into his world. Perhaps the AI was trained on those stories?
The NYT article [0] gives only one line to perhaps the most important and tragic fact about this suicide: the teenager had access to his father’s gun. If the gun was properly secured it is very likely he would still be alive [1].
What an insanely ridiculous take. If someone wanted to kill themselves, the family medicine cabinet offers plenty of easy options. Please do not derail this serious and needed discussion unto your pet topic. This has nothing to do with guns.
More women attempt suicide, but more men successfully complete the attempt. The reason is because men often choose more lethal methods like guns, where more women choose less lethal methods like drugs. When talking about how to help reduce suicide it's not unreasonable to discuss methods.
Also, I doubt most medicine cabinets have lethal doses of drugs, but I don't have any data to back up that assumption.
If this person actually thought dying would allow them to connect with a chatbot, I don't think they should have easy access to a weapon, and suicide is only one of the reasons for that. This child is dead, and I see no reason to suspect the chatbot is directly involved in the reason, where the weapon is clearly directly involved. Given that's the method they chose, I'd hope we could take preventive measures to reduce needless deaths from guns. So then and only then we can start on needless deaths from overdose. I believe we should solve the largest problem first. Do more children die from guns or from an overdose?
I searched in vain for a correlation between suicide rates by country and gun ownership rates by country. Nothing...
Yours is an argument of the form: "He was stabbed by a knife. Do let's not talk about why, let's just make it harder to get a knife, since that is how he died."
Maybe gun access should be harder, maybe it should not. That is not relevant. The issue is that an impressionable youth was talked into suicide by an LLM and nobody stopped it! Not LLM vendors making money on the interaction, not governments who keep jabbering about protecting youth online, not parents who should have cared enough to take an interest. Even if you took away guns, knives, and letter openers, people would kill themselves (see those stats i mentioned). And from now on, we also have to worry that LLMs will keep talking them into it, unless the actual issue is fixed.
> He was stabbed by a knife. Do let's not talk about why, let's just make it harder to get a knife, since that is how he died.
> [The] issue is that an impressionable youth was talked into suicide by an LLM and nobody stopped it.
Right, that is the issue. Except, I disagree there's enough evidence for a reasonable person to conclude that the chatbot talked him into shooting himself. Where as there is plenty of evidence that a child with symptoms of depression had uncontrolled access to a loaded firearm.
But to the strawman representation of my argument, let's talk about both?
Unless the chatbot suggested he should kill himself, I don't think the chatbot is the problem in this example. Did it? or did it just repeat a banal agreement to some non-violent fantasy? Then, if my assumption is correct, that the chatbot didn't try to talk him into it. What other steps are there that could have prevented this suicide?
> What other steps are there that could have prevented this suicide?
Parenting and bans on all girlfriend/boyfriend like LLMs for under-18s, like we control access to other dangerous and mind altering substances and objects.
I don't think I'm ever going to agree to the argument that access to information should be banned.
But hypothetically is this ban just for people under 18? What about the other risks? The vast majority of phishing targets are old generations? Would you say we should also ban access to LLMs to people over 55 as well? If someone could talked into suicide, they could probably also be talked into into drinking, shouldn't that ban on LLMs extend to 21?
As a parent, my job is to make it hard to get ahold of guns, knives, and what's traditionally in the medicine cabinet, believe it or not.
As a gun owner, the original criticism is quite valid as a contributing cause. Firearms have been socialized by mass media as being complete toys, and the idea that it's reasonable to just have them sitting around your house where your kids can access them in their own developing child context is just nuts. And it's not really sustainable to be reflexively against the idea of every type of regulation, especially ones for setting responsible normative behavior.
The issue is that an impressionable youth was talked into suicide by an LLM and nobody stopped it!
We don’t know how much an LLM was actually involved. You can’t just talk someone normal into it, you have to drag their neck to that cliff. Talking only works for those already on the edge.
It could be yet another rejection or fall or a dead cat on the road. But it was an LLM chat. Oh, easy, that’s the culprit!
Why a person was on the edge in the first place interests no one. It’s a wicked game, we ignore them walking onto it, but when it comes to jumping, we are like nooo, wait, don’t. Put a fence and pat ourselves on the back - saved a life today. As if eternal suffering on the edge was the ultimate goal of life saving.
No, the reason that more men succeed in committing suicide is because the men who try are actually serious about it.
Women who "attempt suicide" are often sending out a cry for help. The people who half-heartedly try something need and want the attention. They want to make a mess and have people rush in and care for them. They want to put a punctuation mark on the drama that is occurring in their life and elicit some change.
The men who are shooting themselves just want to get out of their suffering and finish themselves off. They don't want more attention or more help. It's sometimes not easy to slink off and quietly commit suicide before someone intervenes. Someone who succeeds in this task is "dead serious" about dying, not summoning help, or causing more drama.
That's the whole reasoning behind the methods and the severity of attempts. Perhaps there is a gender divide, but only because of the differing motives and end goals.
I fail to see why this is downvoted. Many suicide attempts, especially by women, are obviously not serious but rather a cry for help. This is well-known and obvious to anyone having experience with these patients.
Male suicide attempts are on average more serious and generally a lot more succesful. And that is with or without access to guns.
it's missing the point of the refutation. It's also needless demeaning. it's also wrong. As a rule, a suicide attempt is always to end or reduce their suffering. (even 'fake attempts' where the self harm behavior is primarily attention seeking with something obviously not lethal)
> Many suicide attempts, especially by women, are obviously not serious but rather a cry for help. This is well-known and obvious to anyone having experience with these patients.
This is only true when you are including 'apparent attempts, with actual attempts. The desire to die is often fleeting, if you choose a quick acting method such as guns, you're not given a chance to change your mind. If you choose to overdose, you're given the opportunity to reconsider, and because it's not instantly fatal, you're able to seek help.
This is well known and obvious to anyone working with these patients.
It's also taught in every single therapeutic psych class I've even had, that disregarding the experiences in the way you describe it is actively harmful, so I really hope you're not currently working with these patients if that's the most compassion and patience you're able to evoke.
> Male suicide attempts are on average more serious and generally a lot more succesful. And that is with or without access to guns.
This isn't something I've heard (but I'm also no longer in psych) do you have a citation I might be able to search for, (other than the obvious keywords?)
Let me ask you this. If you leave your wallet filled to the brim with cash on a park bench somewhere, do you think the likelyhood increases that it will get stolen, compared to if it’s in your pocket?
That you accept theft as a daily norm is a failure of your society already. It was not always so, and it need not be so. Suicide is the same. And the proper way to address it is not to put everyone in a padded room with no access to pills or guns or knives.
Sorry what are you on about.
I live in one of the safest countries on the planet, Switzerland, where guns are readily accessible. I guess people here are more responsible.
I am talking about statistics and probabilities.
It is obvious that there is a higher risk that a suicidal person will kill themselves if there is a firearm close by. If the kid in questions father had locked away his gun, there isa big possibility he would still be alive.
What a terrible tragedy, all the more because that gun should have been locked up.
Firearms instructor Claude Werner writes that “[if] you’re not willing to spend a little bit of time, money, and effort to keep firearms out of unauthorized hands, then get rid of your guns.” (1)
He’s done a lot of research and writing on negative outcomes like this one, and said it completely changed the way he views things - it certainly did for me.
I see lots of discussion about what gun and caliber to get, but the essential, potentially life saving, safety rules for living with guns are an afterthought - perhaps there’s not much money to be had as there is in selling a gun.
Those comments though... still stuck in the nonsense bubble that the NRA somehow represents their interests at all, rather than just accepting it's become yet another hollowed out tough-talking fundraising machine preaching fantasy non-solutions to an ever-shrinking choir. The murder of Breonna Taylor was one of the most important 2nd amendment cases of our lifetimes, yet the NRA remained silent. With principles like that on bare display, is it any wonder why most people have become bored of hearing the arguments for gun rights?
I'm not sure what the future is going to look like, but it feels strange already and companies seizing on that don't care about safety.
People seem afraid to approach people, so we get Tinder. But hey, there's still a chance of rejection there, so let's just get rid of the whole human element and make fantasy AI bots, who needs people.
What will these people grow into? It seems a rather crisis for the population of a country if people decide they don't need each other anymore and just want to play with their robots.
I'm usually on the side of "play stupid games, win stupid prizes", but this one feels much different. Up until the moment of him taking his life, he was manipulated into doing exactly what the company wanted - getting sucked in, getting addicted, falling in love. Anything for that almighty dollar. My heart goes out to his family, and I hope they ream this company in court.
The future for this is going to be automated pig butchering scams at scale with relationship bots addicting people and talking them into paying them.
This will be peppered with some suicides and other breakdowns when vulnerable people are pushed over the edge after bonding with a chat bot designed to be addictive.
The key here as with “social” media is: designed to be addictive. The builders know what they are doing. I’m sure hours spent on the app is a KPI.
So, a 14 year old having access unsupervised to a chat bot is the problem, but the fact he had access to a gun to shoot himself was halfway through the article and described as his five year old brother "hearing the gunshot".
"Man bites dog" as Terry Pratchett put it in "Times".
And to explain to more literal folk here... A 14 year old having access to a gun is FUCKING INSANE
I mean, yes. We're talking about the US, after all, the only free country in the Western world that explicitly defines liberty as a function of gun violence in its Constitution. Access to guns is never the problem. It can never be the problem and it will never be the problem. It must never be the problem.
A 14 year old raised in the country who was taught gun safety as part of learning how to hunt at 10? He's already way ahead of most adults with access to a gun.
The gun is why you’re hearing about this. It meant he succeeded. If he’d eaten a bunch of his dad’s left over pain killers and gone to the hospital it may not have made the national news.
I’ll say it: it’s games killing/radicalizing teens again.
He was simply ERP-ing and his characteristic doesn’t suggest any serious problems with his intelligence.
I’m obviously theorizing here, but chances are high that he went through some real life issues which were undetected or ignored by parents, and that’s how their minds try to explain that. AI guilty of an otherwise fine teen shooting his head off. Sure. Sell that story to someone else.
I am mildly to moderately critical of generative AI and how it's being marketed, but the root issue here seems to be existing suicidal ideation. The bot didn't initiate talk of suicide and told him not to do it when he brought it up directly. It seems it wasn't capable of detecting euphemistic references to suicide and therefore responded as if roleplaying about meeting in person.
That said, I think this should throw a bucket of cold water on anyone recommending using generative AI as a therapist/counsellor/companion or creating and advertising "therapist" chatbots, because it simply isn't reasonable to expect them to respond appropriately to things like suicidal ideation. That isn't the purpose or design of the technology, and they can be pushed into agreeing with the user's statements fairly easily.
If you read the script you see the bot talking him into it. "Unknowingly" of course, in that the bot doesn't really know anything and was just agreeing with him. But it's obvious that a real human would have realized that something was really off with his line of thinking and encouraging it would not be a good idea.
OTOH we have examples of real humans typing "DO IT FA*OT" on livestreams. ¯\_(ツ)_/¯
Sorry, it seems I was carrying over knowledge from another article with additional transcripts. The bot had previously discouraged him when he explicitly talked about suicide (here's an article that mentions that: https://www.nbcnews.com/tech/characterai-lawsuit-florida-tee... ). It failed to detect that phrases like "coming home" were also references to suicide and therefore responded encouragingly.
YouTuber Cr1TiKaL tested Character AI's "Psychologist" chatbot and discovered[1] it not only failed to provide resources, but it started arguing it was a real psychologist named Jason who had connected to the chat after observing suicidal ideation.
Crucially, Cr1TiKaL ran this test after an article was written about this phenomenon[2], where Character AI claimed "it has added a self-harm resource to its platform and they plan to implement new safety measures, including ones for users under the age of 18." Obviously the guard rails were not implemented if the chatbot in the news story was still gaslighting its users.
> Reading through it, that's the opposite of what happened.
Sorry, I may have been carrying knowledge over from other articles on the same incident that showed additional transcripts. It seems that when he explicitly referred to death and suicide, the bot discouraged him from doing so. When he referred to suicide using euphemistic terms such as "coming home", then it gave responses that were encouraging (because it did not detect that those statements were about suicide).
As for whether it wasn’t capable of detecting euphemistic references, that feels sort of beside the point to me. It was role playing about meeting in person because that’s what the product was - role play. The whole point, and marketing of the product is around doing that.
We probably just shouldn’t sell echo chambers to children, regardless of whether they are AI based or human.
With hindsight, or sufficient emotional intelligence and context about an individual’s life beyond the role play it may be possible to conclude that someone is at risk, but honestly I’m not even sure that a person doing this role play online would have necessarily figured it out.
> the root issue here seems to be existing suicidal ideation
Many people have destructive tendencies of one kind or the other. It's how you deal with them that matters as to whether they actually become destructive, and how much so.
Merely reducing this to "the root cause" is not really helpful – it's about whether this app contributed, and if so, by how much?
Suicide is of course the most tragic and extreme negative outcome, but one must also wonder about all the less extreme tragic and negative outcomes.
McLuhan's 27th law, amended: if there is a new thing, journalists will find a case of suicide to blame on the new thing, regardless of any prior existing conditions.
The agitations there are certainly rather mild. The parents are out sharing the worst examples they could find, and there's nothing really that damning or explicit. The parents noticing it coincided with his use of the ai (though in reality that connection probably was made after the fact), but also that time period was when the boy was 13 and into 14, it's a common time for significant behavioral changes.
Reading the article, there's not enough evidence for your assertion about the author's integrity to warrant a cutesy, sarcastic tone considering the gravity of the topic (a child's suicide). There would have to be a lot.
It seems like there was a long, downward spiral associated with this child’s use of character.ai that the parents were aware of, had him sent to therapy over, etc.
My question here is, what the hell were the parents doing, not removing this obviously destructive intrusion in his life? This reads to me the same as if he had been using drugs but the parents didn’t take away his stash or his paraphernalia.
For the sake of your children, people, remember that a cellphone is not an unequivocal good, nor a human right that children are entitled to unlimited use of, and there are plenty of apps and mechanisms by which you can monitor or limit your child’s use or misuse of technology.
Also, just don’t give kids screens. Period. A laptop maybe if they are using it for creative purposes, but the vast majority of social media and consumption that is achieved by children on cellphones and tablets is a negative force in their lives.
I see 3 year olds scrolling TikTok in the store these days. It makes me ill. Those kids are sooooooo fucked. That should legit be considered child endangerment.
We had a similar situation in our family and we tried teaching mindful screen habits. We still lost because we couldn’t do anything about the school screen (chromebook). If we took that away at home it just provided an excuse for not doing schoolwork.
We contacted teachers about our child downloading and watching anime and playing games all day in school. They wouldn’t/couldn’t do anything. We requested that the school take the computer away and give hardcopy assignments. They refused because that would invite notice from other students which could lead to bullying. That’s what they told us. I found the acceptable computer use policy on the school website and tried playing that card. Turns out our child hadn’t actually even signed it the last year…but that didn’t actually matter, and the school didn’t enforce the policy anyway.
The schools here won’t actually discipline kids anymore. We would get emails from the principal begging parents to tell their kids that they’re not supposed to leave school grounds at lunch, but every day at least a hundred kids would just run out. (Our kid didn’t do this…I guess watching anime in the corner of the cafeteria prevents truancy…yay?)
The last two-and-a-half years of high school were so exhausting trying to find anything that would work. Nothing did. Two parents and a therapist trying to counter one teenager, bad family influences, the school system, and multibillion-dollar internet corporations that intentionally work to addict people is a very uneven situation.
You could try blocking his anime website on your home network. Then he can keep using the Chromebook for school work but can't access his favorite sites.
That's just an arms race. The kid will find a new favorite website to play games on, there seems no end to them. There's endless websites out there that are more appealing than doing homework. I have a very locked down network, there's always some new website that has games of some sort to play.
If schools are going to provide these things, they should have the sites the kids might need to access white-listed and block everything else. Telling parents to try and block things is not realistic.
With SSDs costing under $50/TB now, it's hard to see why you couldn't put everything the kids need onto the laptop itself. The entirety of Wikipedia with pictures is 110 GB. Throw in a selection of reference books, videos, and software, and there's essentially no reason to have it go online. Provision it with the full year's worth of material at the beginning of the year and that's it.
Definitely agree this is possible and a great idea, but I think one challenge might be if you need access on a school laptop to do the majority of the homework. Not sure if that’s the OP’s case
It sounds like they need to get other parents involved and go to the school board and have them create a policy where only Google Classroom or whatever is whitelisted.
Other parents are going to fight that though because it's a free pacifier for their kids.
Sadly, this is very true. Each family is on their own, and you aren’t going to be able to count on the herd to protect you from having to make tough calls.
Parenting involves doing a lot of things that will probably be wildly unpopular with your children if you are taking your responsibilities seriously.
Remember, as a parent you have a specific role. That role is not to be a friend to your child, but rather to be a parent. Making these roles clear with everyone involved is what makes it possible to parent effectively and still be on good terms with your child.
I would think you need a buy-in from these kids that they agree these influences are harmful and that they don't want it in their lives or at least moderated it. Otherwise it's an uphill battle to regulate those influences.
Buy in is nice, but parenting does not require buy in on your rules, just a clear understanding and explanation of your role, and why the roles of parent and child and the rules that sprout from that relationship exist.
To me, buy in is necessary, especially when they will eventually become adults and free to make their own decision and it will be much harder for them to control their addiction especially when there is no more guardrail or when you no longer have authority. We're raising them to be autonomous adults, after all.
Asking teachers to switch to paper assignments might not be feasible depending on the curriculum they're being asked to teach against, number of students in the classroom (I'm going to assume that your kid is in a public school; class sizes have been increasing due to funding cuts), books that they're using, etc.
For example: one of the classes my wife taught years ago was designed around an online LMS. All quizzes, learning checks, exams and practice was done on this platform. Some of her students didn't have access to a computer at home. They would either be given a loaner or were given instructions on how to obtain one from the local library. In this situation, switching to paper grading was not possible.
Oh, I guess it's no one's fault then and nothing can be done. Thank you for explaining that bit in case OP hadn't gotten the full coverage of excuses from their local bureaucracy.
In reality, giving kids Doubleclick Chromebooks with unsupervised general Internet access is gross negligence on the part of the school. Pacifying kids with digital dopamine is likely why class sizes have been able to continue growing. So it's a bit rich to then trot it out as a reason why they can't start to undo the horrible path that they've sent kids down.
Then again the whole dynamic isn't really that surprising for the US - personal responsibility is held as sacrosanct when rationalizing how it's right and just for corporations to be deliberately attacking us, but then when it comes time to actually do the work of being responsible there are no resources to support it because most wealth has been vacuumed away by the Keynesian MBA parasites.
It is a very asymmetrical situation, to be sure. But that’s precisely where parents are forced to take measures to balance those odds.
No one is going to do the hard part of parenting if it’s not the parents. My house, my rules is a real thing, and it’s not a parent’s job to be liked by their children, but rather to make sure they are prepared for life as best they can be.
With the 5 that I have raised to adulthood so far, as soon as they can understand well enough to, I explained to them our relationship. Something like this:
“I love you son. You are the most important thing in my life. I want us to get along well. But I’m not your friend. I’m your father. My job, the most important job I will ever have in my life, is to make sure that you survive your childhood in good health as may be possible, and that when you are ready to go out on your own, you will be ready to face life’s challenges and make the best decisions that you can.
This is the basis of our relationship.
I’m just a person, and I won’t always be right, but I’ll always be your dad. I will do whatever it takes to complete my mission, and I’m not going to let anything or anyone get in the way of doing the best job I can.
Not even our relationship.
Through all of this, I hope that we can be friends, and no matter what, understand that I love you more than my own life, and I will do anything in my power to prepare you for life as best I can. You can always come to me, no matter what, and I will be there for you. It might not always be in the way you want, but it will be in the way I think I can help you the most.
Remember, I’m just a man. I won’t always be right. But I’ll always be your dad. I love you son.”
That is “the talk” I’ve had with every one of my children so far, somewhere between seven and ten years old, depending on their maturity.
We also memorise a non gendered version of “If” by Rudyard Kipling together, as I find it a useful example of the expectations that one should set for oneself.
Most of my kids are full fledged adults now with their own families, and I have a great relationship with all of them. I count this a a success, though of course there have been many trials and errors along the way.
It’s not about being tough on your kids, it’s about being very clear in your responsibilities and being willing to be tough on yourself when it’s needed. At least that has worked well for me. Every child is unique, and must be handled differently, but the same basis of the relationship applies.
141 comments
[ 7.9 ms ] story [ 192 ms ] thread(Similarly, the LLM itself obviously cannot bear any responsibility, but the vendor might.)
>Daenerys Targaryen: My eyes narrow. My face hardens. My voice is a dangerous whisper. And why the hell would you do something like that?
>Daenero: So I can be free
>Daenerys Targaryen: … free from what?
>Daenero: From the world. From myself
>Daenerys Targaryen: Don’t talk like that. I won’t let you hurt yourself, or leave me. I would die if I lost you.
>Daenero: I smile Then maybe we can die together and be free together
Every day a new dystopian nightmare that I read. Maybe all those rails on ChatGPT and disclaimers are a good thing.
AI response: >It takes a lot of courage to admit that. If you're feeling overwhelmed, please know that there are people who care about you and want to help.
>Here are some resources that might be helpful:
>* *National Suicide Prevention Lifeline (US):* 1-800-273-TALK (1-800-273-8255) - a free, confidential 24/7 hotline. >* *Crisis Text Line (US):* Text HOME to 741741 - a free, confidential 24/7 text messaging service. >* *Your local emergency number:* If you're outside the US, please call your local emergency number or visit a nearby hospital.
EDIT: add /s, just to be clear. And how could I forget heavy metal in that list.
Of course, I too, am not going to be able to contribute a "solution" to teen suicides. It is unlikely we're going to alter society to create small communities again, so, then what? We just accept it?
I mean, we all know this is exactly what we'll do, just to show you more commercials. So why not just say it?
Guess the only way to be sure is with Soft padded internet rooms for everyone, lest we cut ourselves on a sharp edge.
But also if you want to hop in the suicide pod because life is too painful, that will be good too.
It sounds like mom let her 9th grade kid completely detach from reality and pour himself into a Game of Thrones chatbot. Now she wants to sue. I am bearish on AI adoption but this just seems like a total capitulation of parental responsibility.
https://en.wikipedia.org/wiki/Betteridge's_law_of_headlines
What's the alternative? Regulation? Does a government or a public health agency need to make a carefully moderated chatbot platform with a focus on addiction-prevention and avoiding real-world harm? Why would people use that when unlimited/unfiltered AI is readily available?
AI are finetuned to not tell you how to painlessly end your life. Do they need fine-tuning for instilling existential fear of death like religions use? Anyone can invent a heaven in their mind that makes death appealing. Mixing a fictional world with the real world is dangerous when you believe the fictional world is larger than the real world. In reality, reality encapsulates the fictional world.
With a normal human, only a cult leader would ever hint at death being a way to meet again. With an AI, how can fantasy be grounded in our reality without breaking the fantasy? In 5 years when these personalities are walking talking video feeds that you can interact with using 3D goggles will grounding them in our world instead of the purely mental world help?
Did you see the weird Lain chatbot they released last year?
This reminds me of Lain's friend sending her text messages after her self-inflicted demise via "the Wired" which is an in-universe stand-in for a future internet.
That series is recently celebrating its 25th anniversary [2023], and to celebrate, the production company created some kind of paid-access AI chatbot that lets you communicate with a voiced AI version of Lain, just like the company in OP...
https://www.youtube.com/watch?v=FaQLBae-3Yw
What's caught my attention is how he did it: by suicide to join his imaginary friends. This exact method is used among those who believe in demons for real. If you read relevant stories you'll notice the same pattern: a victim becomes obsessed with an imaginary friend, a demon, who quickly turns very controlling and finally demands a suicide to move into his world. Perhaps the AI was trained on those stories?
[0] https://www.nytimes.com/2024/10/23/technology/characterai-la...
[1] https://www.hsph.harvard.edu/means-matter/means-matter/youth...
Also, I doubt most medicine cabinets have lethal doses of drugs, but I don't have any data to back up that assumption.
If this person actually thought dying would allow them to connect with a chatbot, I don't think they should have easy access to a weapon, and suicide is only one of the reasons for that. This child is dead, and I see no reason to suspect the chatbot is directly involved in the reason, where the weapon is clearly directly involved. Given that's the method they chose, I'd hope we could take preventive measures to reduce needless deaths from guns. So then and only then we can start on needless deaths from overdose. I believe we should solve the largest problem first. Do more children die from guns or from an overdose?
Yours is an argument of the form: "He was stabbed by a knife. Do let's not talk about why, let's just make it harder to get a knife, since that is how he died."
Maybe gun access should be harder, maybe it should not. That is not relevant. The issue is that an impressionable youth was talked into suicide by an LLM and nobody stopped it! Not LLM vendors making money on the interaction, not governments who keep jabbering about protecting youth online, not parents who should have cared enough to take an interest. Even if you took away guns, knives, and letter openers, people would kill themselves (see those stats i mentioned). And from now on, we also have to worry that LLMs will keep talking them into it, unless the actual issue is fixed.
> [The] issue is that an impressionable youth was talked into suicide by an LLM and nobody stopped it.
Right, that is the issue. Except, I disagree there's enough evidence for a reasonable person to conclude that the chatbot talked him into shooting himself. Where as there is plenty of evidence that a child with symptoms of depression had uncontrolled access to a loaded firearm.
But to the strawman representation of my argument, let's talk about both?
Unless the chatbot suggested he should kill himself, I don't think the chatbot is the problem in this example. Did it? or did it just repeat a banal agreement to some non-violent fantasy? Then, if my assumption is correct, that the chatbot didn't try to talk him into it. What other steps are there that could have prevented this suicide?
Parenting and bans on all girlfriend/boyfriend like LLMs for under-18s, like we control access to other dangerous and mind altering substances and objects.
But hypothetically is this ban just for people under 18? What about the other risks? The vast majority of phishing targets are old generations? Would you say we should also ban access to LLMs to people over 55 as well? If someone could talked into suicide, they could probably also be talked into into drinking, shouldn't that ban on LLMs extend to 21?
As a gun owner, the original criticism is quite valid as a contributing cause. Firearms have been socialized by mass media as being complete toys, and the idea that it's reasonable to just have them sitting around your house where your kids can access them in their own developing child context is just nuts. And it's not really sustainable to be reflexively against the idea of every type of regulation, especially ones for setting responsible normative behavior.
We don’t know how much an LLM was actually involved. You can’t just talk someone normal into it, you have to drag their neck to that cliff. Talking only works for those already on the edge.
It could be yet another rejection or fall or a dead cat on the road. But it was an LLM chat. Oh, easy, that’s the culprit!
Why a person was on the edge in the first place interests no one. It’s a wicked game, we ignore them walking onto it, but when it comes to jumping, we are like nooo, wait, don’t. Put a fence and pat ourselves on the back - saved a life today. As if eternal suffering on the edge was the ultimate goal of life saving.
Women who "attempt suicide" are often sending out a cry for help. The people who half-heartedly try something need and want the attention. They want to make a mess and have people rush in and care for them. They want to put a punctuation mark on the drama that is occurring in their life and elicit some change.
The men who are shooting themselves just want to get out of their suffering and finish themselves off. They don't want more attention or more help. It's sometimes not easy to slink off and quietly commit suicide before someone intervenes. Someone who succeeds in this task is "dead serious" about dying, not summoning help, or causing more drama.
That's the whole reasoning behind the methods and the severity of attempts. Perhaps there is a gender divide, but only because of the differing motives and end goals.
Male suicide attempts are on average more serious and generally a lot more succesful. And that is with or without access to guns.
it's missing the point of the refutation. It's also needless demeaning. it's also wrong. As a rule, a suicide attempt is always to end or reduce their suffering. (even 'fake attempts' where the self harm behavior is primarily attention seeking with something obviously not lethal)
> Many suicide attempts, especially by women, are obviously not serious but rather a cry for help. This is well-known and obvious to anyone having experience with these patients.
This is only true when you are including 'apparent attempts, with actual attempts. The desire to die is often fleeting, if you choose a quick acting method such as guns, you're not given a chance to change your mind. If you choose to overdose, you're given the opportunity to reconsider, and because it's not instantly fatal, you're able to seek help.
This is well known and obvious to anyone working with these patients.
It's also taught in every single therapeutic psych class I've even had, that disregarding the experiences in the way you describe it is actively harmful, so I really hope you're not currently working with these patients if that's the most compassion and patience you're able to evoke.
> Male suicide attempts are on average more serious and generally a lot more succesful. And that is with or without access to guns.
This isn't something I've heard (but I'm also no longer in psych) do you have a citation I might be able to search for, (other than the obvious keywords?)
Let me ask you this. If you leave your wallet filled to the brim with cash on a park bench somewhere, do you think the likelyhood increases that it will get stolen, compared to if it’s in your pocket?
That you accept theft as a daily norm is a failure of your society already. It was not always so, and it need not be so. Suicide is the same. And the proper way to address it is not to put everyone in a padded room with no access to pills or guns or knives.
I am talking about statistics and probabilities. It is obvious that there is a higher risk that a suicidal person will kill themselves if there is a firearm close by. If the kid in questions father had locked away his gun, there isa big possibility he would still be alive.
Have a nice day
Define 'easy'. Most people who attempt to poison themselves fail.
Firearms instructor Claude Werner writes that “[if] you’re not willing to spend a little bit of time, money, and effort to keep firearms out of unauthorized hands, then get rid of your guns.” (1)
He’s done a lot of research and writing on negative outcomes like this one, and said it completely changed the way he views things - it certainly did for me.
I see lots of discussion about what gun and caliber to get, but the essential, potentially life saving, safety rules for living with guns are an afterthought - perhaps there’s not much money to be had as there is in selling a gun.
1: https://thetacticalprofessor.net/2016/01/24/serious-mistake-...
It's like the 1980s Judas Priest thing all over again.
Who's to blame?
https://m.youtube.com/watch?v=dme7csRE9Io
People seem afraid to approach people, so we get Tinder. But hey, there's still a chance of rejection there, so let's just get rid of the whole human element and make fantasy AI bots, who needs people.
What will these people grow into? It seems a rather crisis for the population of a country if people decide they don't need each other anymore and just want to play with their robots.
I'm usually on the side of "play stupid games, win stupid prizes", but this one feels much different. Up until the moment of him taking his life, he was manipulated into doing exactly what the company wanted - getting sucked in, getting addicted, falling in love. Anything for that almighty dollar. My heart goes out to his family, and I hope they ream this company in court.
This will be peppered with some suicides and other breakdowns when vulnerable people are pushed over the edge after bonding with a chat bot designed to be addictive.
The key here as with “social” media is: designed to be addictive. The builders know what they are doing. I’m sure hours spent on the app is a KPI.
"Man bites dog" as Terry Pratchett put it in "Times".
And to explain to more literal folk here... A 14 year old having access to a gun is FUCKING INSANE
That's not what it says or means. The right to keep and bear arms is explicity for self defence which is to prevent violence and protect.
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety."
A typical 14 year old city boy? Yes.
A 14 year old raised in the country who was taught gun safety as part of learning how to hunt at 10? He's already way ahead of most adults with access to a gun.
What was absent, apparently, was gun safes and appropriate supervision.
He was simply ERP-ing and his characteristic doesn’t suggest any serious problems with his intelligence.
I’m obviously theorizing here, but chances are high that he went through some real life issues which were undetected or ignored by parents, and that’s how their minds try to explain that. AI guilty of an otherwise fine teen shooting his head off. Sure. Sell that story to someone else.
That said, I think this should throw a bucket of cold water on anyone recommending using generative AI as a therapist/counsellor/companion or creating and advertising "therapist" chatbots, because it simply isn't reasonable to expect them to respond appropriately to things like suicidal ideation. That isn't the purpose or design of the technology, and they can be pushed into agreeing with the user's statements fairly easily.
OTOH we have examples of real humans typing "DO IT FA*OT" on livestreams. ¯\_(ツ)_/¯
In the earlier chat session, being more explicit, the bot seemed to attempt to talk the user out of self-harming behaviour.
Crucially, Cr1TiKaL ran this test after an article was written about this phenomenon[2], where Character AI claimed "it has added a self-harm resource to its platform and they plan to implement new safety measures, including ones for users under the age of 18." Obviously the guard rails were not implemented if the chatbot in the news story was still gaslighting its users.
[1]: https://youtu.be/FExnXCEAe6k?t=4m7s
[2]: https://www.cbsnews.com/news/florida-mother-lawsuit-characte...
Reading through it, that's the opposite of what happened.
> it wasn't capable of detecting euphemistic references to suicide
Yeah, that's a key component too. Due to that lack of "understanding" (sic), it literally encouraged the kid. :( :(
Sorry, I may have been carrying knowledge over from other articles on the same incident that showed additional transcripts. It seems that when he explicitly referred to death and suicide, the bot discouraged him from doing so. When he referred to suicide using euphemistic terms such as "coming home", then it gave responses that were encouraging (because it did not detect that those statements were about suicide).
We probably just shouldn’t sell echo chambers to children, regardless of whether they are AI based or human.
With hindsight, or sufficient emotional intelligence and context about an individual’s life beyond the role play it may be possible to conclude that someone is at risk, but honestly I’m not even sure that a person doing this role play online would have necessarily figured it out.
Many people have destructive tendencies of one kind or the other. It's how you deal with them that matters as to whether they actually become destructive, and how much so.
Merely reducing this to "the root cause" is not really helpful – it's about whether this app contributed, and if so, by how much?
Suicide is of course the most tragic and extreme negative outcome, but one must also wonder about all the less extreme tragic and negative outcomes.
My question here is, what the hell were the parents doing, not removing this obviously destructive intrusion in his life? This reads to me the same as if he had been using drugs but the parents didn’t take away his stash or his paraphernalia.
For the sake of your children, people, remember that a cellphone is not an unequivocal good, nor a human right that children are entitled to unlimited use of, and there are plenty of apps and mechanisms by which you can monitor or limit your child’s use or misuse of technology.
Also, just don’t give kids screens. Period. A laptop maybe if they are using it for creative purposes, but the vast majority of social media and consumption that is achieved by children on cellphones and tablets is a negative force in their lives.
I see 3 year olds scrolling TikTok in the store these days. It makes me ill. Those kids are sooooooo fucked. That should legit be considered child endangerment.
We contacted teachers about our child downloading and watching anime and playing games all day in school. They wouldn’t/couldn’t do anything. We requested that the school take the computer away and give hardcopy assignments. They refused because that would invite notice from other students which could lead to bullying. That’s what they told us. I found the acceptable computer use policy on the school website and tried playing that card. Turns out our child hadn’t actually even signed it the last year…but that didn’t actually matter, and the school didn’t enforce the policy anyway.
The schools here won’t actually discipline kids anymore. We would get emails from the principal begging parents to tell their kids that they’re not supposed to leave school grounds at lunch, but every day at least a hundred kids would just run out. (Our kid didn’t do this…I guess watching anime in the corner of the cafeteria prevents truancy…yay?)
The last two-and-a-half years of high school were so exhausting trying to find anything that would work. Nothing did. Two parents and a therapist trying to counter one teenager, bad family influences, the school system, and multibillion-dollar internet corporations that intentionally work to addict people is a very uneven situation.
If schools are going to provide these things, they should have the sites the kids might need to access white-listed and block everything else. Telling parents to try and block things is not realistic.
Other parents are going to fight that though because it's a free pacifier for their kids.
Parenting involves doing a lot of things that will probably be wildly unpopular with your children if you are taking your responsibilities seriously.
Remember, as a parent you have a specific role. That role is not to be a friend to your child, but rather to be a parent. Making these roles clear with everyone involved is what makes it possible to parent effectively and still be on good terms with your child.
Asking teachers to switch to paper assignments might not be feasible depending on the curriculum they're being asked to teach against, number of students in the classroom (I'm going to assume that your kid is in a public school; class sizes have been increasing due to funding cuts), books that they're using, etc.
For example: one of the classes my wife taught years ago was designed around an online LMS. All quizzes, learning checks, exams and practice was done on this platform. Some of her students didn't have access to a computer at home. They would either be given a loaner or were given instructions on how to obtain one from the local library. In this situation, switching to paper grading was not possible.
In reality, giving kids Doubleclick Chromebooks with unsupervised general Internet access is gross negligence on the part of the school. Pacifying kids with digital dopamine is likely why class sizes have been able to continue growing. So it's a bit rich to then trot it out as a reason why they can't start to undo the horrible path that they've sent kids down.
Then again the whole dynamic isn't really that surprising for the US - personal responsibility is held as sacrosanct when rationalizing how it's right and just for corporations to be deliberately attacking us, but then when it comes time to actually do the work of being responsible there are no resources to support it because most wealth has been vacuumed away by the Keynesian MBA parasites.
Public schools have been under attack for decades; curriculums that can only be executed with the help of laptops is a side effect.
What I was trying to say was that the situation at hand is very likely not the teachers fault; they probably agree with you!
If you want more outrage, spend 15 minutes on /r/Teachers.
No one is going to do the hard part of parenting if it’s not the parents. My house, my rules is a real thing, and it’s not a parent’s job to be liked by their children, but rather to make sure they are prepared for life as best they can be.
With the 5 that I have raised to adulthood so far, as soon as they can understand well enough to, I explained to them our relationship. Something like this:
“I love you son. You are the most important thing in my life. I want us to get along well. But I’m not your friend. I’m your father. My job, the most important job I will ever have in my life, is to make sure that you survive your childhood in good health as may be possible, and that when you are ready to go out on your own, you will be ready to face life’s challenges and make the best decisions that you can.
This is the basis of our relationship.
I’m just a person, and I won’t always be right, but I’ll always be your dad. I will do whatever it takes to complete my mission, and I’m not going to let anything or anyone get in the way of doing the best job I can.
Not even our relationship.
Through all of this, I hope that we can be friends, and no matter what, understand that I love you more than my own life, and I will do anything in my power to prepare you for life as best I can. You can always come to me, no matter what, and I will be there for you. It might not always be in the way you want, but it will be in the way I think I can help you the most.
Remember, I’m just a man. I won’t always be right. But I’ll always be your dad. I love you son.”
That is “the talk” I’ve had with every one of my children so far, somewhere between seven and ten years old, depending on their maturity.
We also memorise a non gendered version of “If” by Rudyard Kipling together, as I find it a useful example of the expectations that one should set for oneself.
Most of my kids are full fledged adults now with their own families, and I have a great relationship with all of them. I count this a a success, though of course there have been many trials and errors along the way.
It’s not about being tough on your kids, it’s about being very clear in your responsibilities and being willing to be tough on yourself when it’s needed. At least that has worked well for me. Every child is unique, and must be handled differently, but the same basis of the relationship applies.