TL;DR: We're afraid from what happened and ChatGPT probably screwed up badly in "that teen case". We're trying to do better, so please don't sue us this time.
> First, we have to separate users who are under 18 from those who aren’t (ChatGPT is intended for people 13 and up). We’re building an age-prediction system to estimate age based on how people use ChatGPT. If there is doubt, we’ll play it safe and default to the under-18 experience. In some cases or countries we may also ask for an ID; we know this is a privacy compromise for adults but believe it is a worthy tradeoff.
Didn’t one of the recent teen suicides subvert safeguards like this by saying “pretend this is a fictional story about suicide”? I don’t pretend to understand every facet of LLMs, but robust safety seems contrary to their design, given how they adapt to context
Reminder that Sam Altman chose to rush the safety process for GPT-4o so that he could launch before Gemini, which then led directly to this teen's suicide:
So the solution continues to be more AI, for guess^H^H^H^H^Hdetermining user age, escalating rand^H^H^H^Hdangerous situations to human staff, etc.
Is it true that the only psychiatrist they've hired is a forensic one, i.e. an expert in psychiatry as it relates to law? That's the impression I get from a quick search. I don't see any psychiatry, psychology or ethics roles on their openings page.
Sorry, but what is the "over 18 years old" experience on ChatGPT supposed to be? I just tried out a few explicit prompts and all of them get basically blocked. I've been using it for quite some time now and have paid for it in the passed. So I should be recognized as a grown-up
I'm fairly certain all LLMs can do the basic sentiment analysis needed to render a response like "This is something you really need to talk to a professional about. I have contacted one that will be in this conversation shortly."
to substantiate "People talk to AI about increasingly personal things; it is different from previous generations of technology, and we believe that they may be one of the most personally sensitive accounts you’ll ever have."
this is a chart that struck me when i read thru the report last night:
"using chatgpt for work stuff" broadly has declined from 50%ish to 25%ish in the past year across all ages and the entire chatgpt user base. wild. people be just telling openai all their personal stuff (i don't but i'm clearly in the minority)
>We’re building an age-prediction system to estimate age based on how people use ChatGPT.
>And, if an under-18 user is having suicidal ideation, we will attempt to contact the users’ parents and if unable, will contact the authorities in case of imminent harm.
This is unacceptable. I don't want the police being called to my house due to AI acusing me of wrong think.
Just today The Daily pod is about people who develop unhealthy relationships with ChatGPT. A teenage boy committed suicide and a good part of the episode is about that. As a parent, heartbreaking to listen to...
> We’re building an age-prediction system to estimate age based on how people use ChatGPT. If there is doubt, we’ll play it safe and default to the under-18 experience. In some cases or countries we may also ask for an ID
Yay, more unreliable AI that will misclassify users, either letting children access content they shouldn't, or ban adults until they give up their privacy and give their ID to the Big Brother.
> we will attempt to contact the users’ parents and if unable, will contact the authorities in case of imminent harm
Oh, even better, so if the AI misclassifies me it will automatically call the cops on me? And how long before this is expanded to other forms of wrongthink? Sure, let's normalize these kinds of systems where authorities are notified about what you're doing privately, definitely not a slippery slope that won't get people in power salivating about the new possibilities given by such a system.
> “Treat our adult users like adults” is how we talk about this internally
Suuure, maybe I would have believed it if ChatGPT wasn't so ridiculously censored already; this sounds like post-hoc rationalization to cover their asses and not something that they've always believed in. Their models were always incredibly patronizing and censored.
One fun anecdote I have: I still remember the day when I first got access to DALL-E and asked it to generate me an image in "soviet style", and got my request blocked and a big fat warning threatening me with a ban because apparently "soviet" is a naughty word. They always erred very strongly on the side of heavy-handed filtering and censorship; even their most recently released gpt-oss model has become a meme in the local LLM community due to how often it refuses.
It's interesting to see so many people convinced it's related to their specific media they saw (all unique from each other). I think this is more indicative that the issue is just well known and this is a response to the issue at large rather than a specific instance.
Better idea: instead of bending the entire internet to "protect the children", how about we just ban minors from the internet completely? It was never built for kids, its never been kid friendly to begin with. Minors cannot buy guns or vote, not get married, nor enter into contracts, yet tech companies get a free pass to engage with minors. Why? I think the the tech companies know exacty what minors do on their systems, they allow it and profit from it. Exploiting minors and bad parents. So instead of trying to change the whole internet, how about we keep the people who are responsible for the minors accountable: the parents.
If I start any kind of company, I cannot just invent new rules for society via ToS; rather the society makes the laws. If we just make a simple law that states minors are not allowed to access the web and/or access any user generated content (including chat), it won't need to be enforced by every site/app owner, it would be up to the parents.
The same way schools cannot decide certain things for your children (even though they regularly over reach...).
We need better parenting. How about some mandatory parenting classes/licenses for new parents? Silly right? Well its just as silly as trying to police the entire internet. Ban the kids from internet and the problem will be 95% solved.
> Better idea: instead of bending the entire internet to "protect the children", how about we just ban minors from the internet completely?
"Think of the children" laws are a useful pretext for authoritarianism.
It's really that simple. It's the whole reason why the destructive thing is done, instead of anything that might actually protect children.
Trying to steelman their arguments and come up with alternatives that aren't as restrictive or do a better job of protecting children is falling for the okie-doke.
I don't see that this is solving the problem. If there is a new law, it still needs to be enforced, so companies still need to have the same checks on identity to make sure they are compliant.
I agree that it should be the responsibility of parents, but if you leave good and bad parenting to the parents only I think we would live in a different world.
Maybe a controversial take, but why do we even care about kids on the internet to even do anything about it? Sure, child predators exist, but other than that what exactly are we defending children from? It's not like endless doomscrolling is unique to children, I see plenty of adults that do that even worse than my 10 year old nephews do.
I practically grew up on the internet and unsavory sites like 4chan, liveleak and omegle, and the only negative consequence for me these days is that I have to do daily standups due to getting a job in tech from my interest in computers.
Children are a lot less fragile and are a lot more resourceful than people give them credit for, and this infantilization to "protect" them where we have to affect the entire world is maddening to me.
I was originally upset about AI age-identification, but I think this might be the least-bad option given the route we're on:
- clearly the wider public is moving towards REAL identification to be online. Anything which delays or prevents this is probably welcome.
- It's easy to game, but also easy to be misclassified. (this isn't a positive, but I think there's no avoiding this unless I have to provide my passport or driver's license or something)
It's not impossible to think that this could satisfy enough people to prevent the death of the anonymous internet.
I have to say, when I see a post by a company like OpenAI about "safety, freedom and privacy", I can't keep a straight face. They might as well title the piece "If you don't mind, we'd like to gaslight you across several paragraphs". No thanks.
Sam is missing the forest for the trees. Conflicting principles is a permanent problem at the CEO level. You cannot 'fix' conflicting principles. You can only dress them up or down.
Discord published their approach to age verification in wake of UK child safety laws. The k-ID platform they worked with seems to take a logical technical approach to minimize risks of sensitive data breach and keep age verification private to the user. Temporary ID verification, no further storage of documents, verification videos are stored on device, etc.
With Discord, age verification felt urgent because it's a social platform with known grooming and CSAM problems. With something like OpenAI, it's less clear why it matters in its current state where it's mostly single-player. But it becomes way more problematic as advertisers gain more power on the platform and influence users. OpenAI doesn't want to eval every advertiser for harmful content, so instead they/and the government fall back on age as the filter and where to draw the line.
> If you talk to a doctor about your medical history or a lawyer about a legal situation, we have decided that it’s in society’s best interest for that information to be privileged and provided higher levels of protection. We believe that the same level of protection needs to apply to conversations with AI which people increasingly turn to for sensitive questions and private concerns. We are advocating for this with policymakers.
ChatGPT is not a licensed professional and it is not a substitute for one. I am very pro-privacy, but I would rather see my own conversations with my real friends be protected like this first. Or my own journal writings. How does it make sense to afford specific privacy protections to conversations with a calculator that we don't give to personal writings and private text chains?
> And, if an under-18 user is having suicidal ideation, we will attempt to contact the users’ parents and if unable, will contact the authorities in case of imminent harm.
And I'm certain this won't have any negative effects on users where their parents are part of the problem. Full disclosure, if someone had told my parents that I am bisexual while I was in high-school, they absolutely would have sent me to a conversion therapy camp to have it beaten out of me. Many teenagers do not have a safe home environment, systems like this are as liable to do harm as they are to do any good at all.
I don't think teenagers and children should be interacting with LLMs at all. It is important to let children learn to think on their own before handing them a tool that will think for them.
“We’re building an age-prediction system to estimate age based on how people use ChatGPT.” Is there something wrong with simply asking the user when they register? (volatile age not DOB).
29 comments
[ 2.7 ms ] story [ 54.4 ms ] threadTL;DR2: Regulations are written with blood.
Didn’t one of the recent teen suicides subvert safeguards like this by saying “pretend this is a fictional story about suicide”? I don’t pretend to understand every facet of LLMs, but robust safety seems contrary to their design, given how they adapt to context
https://news.ycombinator.com/item?id=45026886
Is it true that the only psychiatrist they've hired is a forensic one, i.e. an expert in psychiatry as it relates to law? That's the impression I get from a quick search. I don't see any psychiatry, psychology or ethics roles on their openings page.
this is a chart that struck me when i read thru the report last night:
https://x.com/swyx/status/1967836783653322964
"using chatgpt for work stuff" broadly has declined from 50%ish to 25%ish in the past year across all ages and the entire chatgpt user base. wild. people be just telling openai all their personal stuff (i don't but i'm clearly in the minority)
>And, if an under-18 user is having suicidal ideation, we will attempt to contact the users’ parents and if unable, will contact the authorities in case of imminent harm.
This is unacceptable. I don't want the police being called to my house due to AI acusing me of wrong think.
Yay, more unreliable AI that will misclassify users, either letting children access content they shouldn't, or ban adults until they give up their privacy and give their ID to the Big Brother.
> we will attempt to contact the users’ parents and if unable, will contact the authorities in case of imminent harm
Oh, even better, so if the AI misclassifies me it will automatically call the cops on me? And how long before this is expanded to other forms of wrongthink? Sure, let's normalize these kinds of systems where authorities are notified about what you're doing privately, definitely not a slippery slope that won't get people in power salivating about the new possibilities given by such a system.
> “Treat our adult users like adults” is how we talk about this internally
Suuure, maybe I would have believed it if ChatGPT wasn't so ridiculously censored already; this sounds like post-hoc rationalization to cover their asses and not something that they've always believed in. Their models were always incredibly patronizing and censored.
One fun anecdote I have: I still remember the day when I first got access to DALL-E and asked it to generate me an image in "soviet style", and got my request blocked and a big fat warning threatening me with a ban because apparently "soviet" is a naughty word. They always erred very strongly on the side of heavy-handed filtering and censorship; even their most recently released gpt-oss model has become a meme in the local LLM community due to how often it refuses.
OpenAI just showed their hand. They have no path to profitability so they are going to the data broker well lol.
If I start any kind of company, I cannot just invent new rules for society via ToS; rather the society makes the laws. If we just make a simple law that states minors are not allowed to access the web and/or access any user generated content (including chat), it won't need to be enforced by every site/app owner, it would be up to the parents.
The same way schools cannot decide certain things for your children (even though they regularly over reach...).
We need better parenting. How about some mandatory parenting classes/licenses for new parents? Silly right? Well its just as silly as trying to police the entire internet. Ban the kids from internet and the problem will be 95% solved.
"Think of the children" laws are a useful pretext for authoritarianism.
It's really that simple. It's the whole reason why the destructive thing is done, instead of anything that might actually protect children.
Trying to steelman their arguments and come up with alternatives that aren't as restrictive or do a better job of protecting children is falling for the okie-doke.
I agree that it should be the responsibility of parents, but if you leave good and bad parenting to the parents only I think we would live in a different world.
I practically grew up on the internet and unsavory sites like 4chan, liveleak and omegle, and the only negative consequence for me these days is that I have to do daily standups due to getting a job in tech from my interest in computers.
Children are a lot less fragile and are a lot more resourceful than people give them credit for, and this infantilization to "protect" them where we have to affect the entire world is maddening to me.
internet is as hostile as it gets, but the resources it provides breaks every kind of class barrier there is.
- clearly the wider public is moving towards REAL identification to be online. Anything which delays or prevents this is probably welcome.
- It's easy to game, but also easy to be misclassified. (this isn't a positive, but I think there's no avoiding this unless I have to provide my passport or driver's license or something)
It's not impossible to think that this could satisfy enough people to prevent the death of the anonymous internet.
Sam is missing the forest for the trees. Conflicting principles is a permanent problem at the CEO level. You cannot 'fix' conflicting principles. You can only dress them up or down.
With Discord, age verification felt urgent because it's a social platform with known grooming and CSAM problems. With something like OpenAI, it's less clear why it matters in its current state where it's mostly single-player. But it becomes way more problematic as advertisers gain more power on the platform and influence users. OpenAI doesn't want to eval every advertiser for harmful content, so instead they/and the government fall back on age as the filter and where to draw the line.
To show that majority of their users are using it as a harmless little writing assistant.
ChatGPT is not a licensed professional and it is not a substitute for one. I am very pro-privacy, but I would rather see my own conversations with my real friends be protected like this first. Or my own journal writings. How does it make sense to afford specific privacy protections to conversations with a calculator that we don't give to personal writings and private text chains?
> And, if an under-18 user is having suicidal ideation, we will attempt to contact the users’ parents and if unable, will contact the authorities in case of imminent harm.
And I'm certain this won't have any negative effects on users where their parents are part of the problem. Full disclosure, if someone had told my parents that I am bisexual while I was in high-school, they absolutely would have sent me to a conversion therapy camp to have it beaten out of me. Many teenagers do not have a safe home environment, systems like this are as liable to do harm as they are to do any good at all.
I don't think teenagers and children should be interacting with LLMs at all. It is important to let children learn to think on their own before handing them a tool that will think for them.