"Going forward, it will use technology to detect underage users based on conversations and interactions on the platform, as well as information from any connected social media accounts"
Something tells me that this ain't gonna work. Kids and teens are more inventive than they probably realize.
I don't think the "use technology to detect underage users" approach has ever worked well for its stated intent (it works okay for being able to say you're "doing something" about the problem while not losing as many users), but it's slightly better than mandatory ID for everyone.
Mandatory ID wouldn’t be that bad if implemented claims-based, from within the browser.
So that the browser could ask:
Hey government, is this user above 18?
And the government just responds with yes or no.
Sites would then only need to whitelist browsers with this check an merely see an “allowed” claim for whatever type of site they have.
Parents or users could use parental controls to “allow” or “block” certain categories, meaning website creators wouldn’t even get any info on the user.
They don't care: if they put in robust sounding guards and openly state "chatbots are not for kids", they can point to them in court if someone gets around it
Nah, you just have to have some prompt that unknowingly reveals that the user has ever engaged with Roblox in any way, which leads to an instant block.
I would expect that "based on conversations and interactions on the platform" would result in both false positives and false negatives, and that "information from any connected social media accounts" might cause additional problems (possibly including that some people being unable to get in at all, or being able to get in once and then unable to access for reasons that are not explained). (It won't affect me because I do not use it, but it might affect someone who does try to use it)
This is not in the spirit of this law, and will probably irk some, but...
If a kid thinks like an adult, behaves like an adult and can't be distinguished from an adult from their online presence, let them use the chatbot.
On the other end, I wish they'd flag immature adults as kids as well.
18 is an arbitrary number, and if we have more appropriate ways to judge if someone is ready or not (assuming their check is worth its salt), it should be fine to defer to that.
It's not like they're going to a bar to do tequila shots or scam retirees for insurance money.
Finally. CAI is notoriously shady with harvesting, selling, and making it difficult to remove data. How many kids have CAI's chatbots already seduced and had their intimate conversations sold to Google?
I suspect Google is the one pushing for this. C.ai has dramatically folded on two of its longest standing struggles in the last two months: underage users and intellectual property
In both cases they went nuclear in a way that implies they actually don't care if the current product survives as long as C.ai (read: Google) isn't exposed to the ongoing risk
Very nice. Just yesterday I wrote about the 13-18 age group using ChatGPT and how I think it should be disallowed (without guardian consent), this was in the context of suicide cases.
On a similar note, I was completing my application for YC Startup School / Co-Founder matching program. And when listing possible ideas for startups I straight out explicitly mentioned I'm not interested in pursuing AI ideas at the moment, AI features are fine, but not as the main pitch.
It feels like at least for me the bubble has popped, I have talked also recently about the way in which the bubble might pop would be due to legal liability collapse in the courts. https://news.ycombinator.com/item?id=45727060
This added with the fact that AI was always a vague folk category of software, it's being used for robotics, NLP and fake images, I just don't think it's a real taxon.
Similar to the crypto buzz from the last season, the reputable parties will exit and stop associating, while the grifters and free-associating mercenaries will remain.
Even if you are completely selfish, it's not even hugely beneficial to be in the "AI" space, at least in my experience, customers come in with huge expectations, and non-huge budgets. Even if you sell your soul to implement a chatbot that will replace 911 operators, at this point the major actors have already done so, or not, and you are left with small companies that want to be able to fire 5 employees and will pay you 3 months of employee salary if you can get it done by vibe code completing their vibe coded prototype within a 2-3 deadline.
If parental consent is all that's necessary then you might as well blanket allow it. I'm a parent and interacting with a lot of parents of young children, the vast majority of them can't be bothered to deal with whining or inconvenience. Which is why I see 8 year olds with cash app even though that's not allowed any many abandon or extremely weaken parental controls on basically every advice.
My personal favorite story is when I talk my youngest aunt about a videogame my cousin wanted. She said no absolutely not. Then proceeded to buy the game for ehr 10 year old. A game she was carded for. A game that has that it's M rated and has adult themes and whatnot on the box. She called me later in horror about how inappropriate this game I told her about 2 weeks earlier was. How could they make games like that for children she says about the game she was carded for because it's only for adults.
I use her as an example but that situation is a lot of parents. I personally think that it's not the government's place to say how much exposure I want to give my child to the internet, but I have rules and boundaries around that with my kid. Many of her friends have free access and and have always had it since toddlers. People say it's parents not being savvy, but honestly it's parents not caring. Parent controls have been around over 30 years and they have always been dead simple. But they do increase the whining in your life from your kids and that means if parents can allow it a high quantity will. I have no faith that a law will stop significantly more kids than no law. I know too many parents who allow their kids to do things they know are harmful to their kid because "I don't want them to feel left out" or they don't want to deal with whining.
This is the only way. Tech companies cannot become like the police in many countries. The police in many countries are used as a "catch-all" function at the moment of, where they have to deal with the failure of other functions (parenting, community) exactly at the moment of (just-in time catch-all).
Your kid is on the fucking computer all day building an unhealthy relationship with essentially a computer game character. Step the fuck in. These companies absolutely have to make liability clear here. It's an 18+ product, watch your kids.
I worry that all this does is reduce future liability issues and make those users use another chatbot instead. I trust character.ai more than other chatbots for safety.
> [Dr. Nina Vasan] said the company should work with child psychologists and psychiatrists to understand how suddenly losing access to A.I. companions would affect young users.
I hope there’s a more gradual transition here for those users. AI companions are often far more available than other people, so it’s easy to talk more and get more attached to them. This restriction may end up being a net negative to affected users.
It is also highly compatible with the internet both in terms of technical/performance scalability and utility scalability (you can use it for just about any information verification need in any kind of application).
How about we just enact a law that state children aren't allowed online, not allowed to interact with interative software (like ai, user generated content). Make the parents responsible, not companies.
I think ChatBots are not very helpful if you don't treat them with scrutiny. There should be a mandatory requirement of a disclaimer before interacting with all AI.
Lots of threads in here saying this is just a "legal" protection move...
I'd like to believe that most actual people want to protect kids.
It's easy to write off corporations and forget that they are founded by real people and employ real people... some with kids of their own or with nieces or nephews etc, and some of them probably do really care.
Not saying character.ai is driven by that but I imagine the times they've been in the news were genuinely hard times to be working there...
There is nothing about character.ai that would be appealing for more than 30 minutes to anybody that isn't suffering from some kind of acute mental health crisis. They should shut the whole thing down. That is a cool demo that should have never been made into a product.
29 comments
[ 2.2 ms ] story [ 58.3 ms ] threadTeen in love with chatbot killed himself – can the chatbot be held responsible?
https://news.ycombinator.com/item?id=45726556
Something tells me that this ain't gonna work. Kids and teens are more inventive than they probably realize.
So that the browser could ask: Hey government, is this user above 18? And the government just responds with yes or no.
Sites would then only need to whitelist browsers with this check an merely see an “allowed” claim for whatever type of site they have.
Parents or users could use parental controls to “allow” or “block” certain categories, meaning website creators wouldn’t even get any info on the user.
If a kid thinks like an adult, behaves like an adult and can't be distinguished from an adult from their online presence, let them use the chatbot. On the other end, I wish they'd flag immature adults as kids as well.
18 is an arbitrary number, and if we have more appropriate ways to judge if someone is ready or not (assuming their check is worth its salt), it should be fine to defer to that.
It's not like they're going to a bar to do tequila shots or scam retirees for insurance money.
In both cases they went nuclear in a way that implies they actually don't care if the current product survives as long as C.ai (read: Google) isn't exposed to the ongoing risk
https://news.ycombinator.com/item?id=45733618
On a similar note, I was completing my application for YC Startup School / Co-Founder matching program. And when listing possible ideas for startups I straight out explicitly mentioned I'm not interested in pursuing AI ideas at the moment, AI features are fine, but not as the main pitch.
It feels like at least for me the bubble has popped, I have talked also recently about the way in which the bubble might pop would be due to legal liability collapse in the courts. https://news.ycombinator.com/item?id=45727060
This added with the fact that AI was always a vague folk category of software, it's being used for robotics, NLP and fake images, I just don't think it's a real taxon.
Similar to the crypto buzz from the last season, the reputable parties will exit and stop associating, while the grifters and free-associating mercenaries will remain.
Even if you are completely selfish, it's not even hugely beneficial to be in the "AI" space, at least in my experience, customers come in with huge expectations, and non-huge budgets. Even if you sell your soul to implement a chatbot that will replace 911 operators, at this point the major actors have already done so, or not, and you are left with small companies that want to be able to fire 5 employees and will pay you 3 months of employee salary if you can get it done by vibe code completing their vibe coded prototype within a 2-3 deadline.
My personal favorite story is when I talk my youngest aunt about a videogame my cousin wanted. She said no absolutely not. Then proceeded to buy the game for ehr 10 year old. A game she was carded for. A game that has that it's M rated and has adult themes and whatnot on the box. She called me later in horror about how inappropriate this game I told her about 2 weeks earlier was. How could they make games like that for children she says about the game she was carded for because it's only for adults.
I use her as an example but that situation is a lot of parents. I personally think that it's not the government's place to say how much exposure I want to give my child to the internet, but I have rules and boundaries around that with my kid. Many of her friends have free access and and have always had it since toddlers. People say it's parents not being savvy, but honestly it's parents not caring. Parent controls have been around over 30 years and they have always been dead simple. But they do increase the whining in your life from your kids and that means if parents can allow it a high quantity will. I have no faith that a law will stop significantly more kids than no law. I know too many parents who allow their kids to do things they know are harmful to their kid because "I don't want them to feel left out" or they don't want to deal with whining.
Your kid is on the fucking computer all day building an unhealthy relationship with essentially a computer game character. Step the fuck in. These companies absolutely have to make liability clear here. It's an 18+ product, watch your kids.
> [Dr. Nina Vasan] said the company should work with child psychologists and psychiatrists to understand how suddenly losing access to A.I. companions would affect young users.
I hope there’s a more gradual transition here for those users. AI companions are often far more available than other people, so it’s easy to talk more and get more attached to them. This restriction may end up being a net negative to affected users.
https://news.ycombinator.com/item?id=44723418
It is also highly compatible with the internet both in terms of technical/performance scalability and utility scalability (you can use it for just about any information verification need in any kind of application).
https://www.bbc.co.uk/news/technology-67012224
I'd like to believe that most actual people want to protect kids.
It's easy to write off corporations and forget that they are founded by real people and employ real people... some with kids of their own or with nieces or nephews etc, and some of them probably do really care.
Not saying character.ai is driven by that but I imagine the times they've been in the news were genuinely hard times to be working there...