The title and story are rather misleading, he's been charged with “exposing a child to harmful material” and “sexual contact with a child under age 13" rather than for generating the images per-se.
It's confusing because that was also my interpretation until I saw the quote from the DoJ prosecutor in the article:
>“Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material"
I completely understand forbidding the distribution to minors, but how can they hold accountable someone for producing CSAM and keeping all data in their hard drive?
This sets a dangerous precedent; how is it different from simulating a murder, or simply speeding in a videogame?
That "or distribute" should really be "and distribute"; the whole point should be to protect kids, not to punish people that are indeed fucked up but found a way to not harm the society.
Running some order of magnitudes, we've had people abusing children and taking photos of it since around 1850. By this point there is a large enough corpus of whatever horrors they want to look at that a model can be trained without doing any further harm.
It is a total waste of taxpayer funds to be going after someone if all they've done is generate however many images with stable diffusion. I'm glad in this case he's at least done something objectionable although an arrest seems a bit of an overreach, this seems to be the sort of situation that could be resolved with a fine rather than jail time based on skimming the indictment. If that.
How would policing that work exactly? Downloading only old CSAM is acceptable? Technically any child abuse material available was produced in the past, so you could argue any pedophile downloading any of it isn't 'doing any further harm'.
But the harm in it comes from generating further demand and normalising the sexualization of children. And than can happen with any material from before or after 1850. If a model is trained on it, it becomes a derivative work that should be treated no differently than a b&w version of a colour abuse image.
Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service.
> Technically any child abuse material available was produced in the past, so you could argue any pedophile downloading any of it isn't 'doing any further harm'.
Sounds like I'd be happy to go with that. You're taking it to the logical conclusion and I generally agree. The problem to me is (1) people abusing children and (2) people paying other people to abuse children.
> Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service.
It isn't an important service. With the advent of generative AI models, going after people for possessing dodgy images is now more than likely persecuting people who haven't done any harm to anyone and aren't going to.
We really should have learned this lesson after Turing. If people aren't harming anyone then it is OK for them to have bizarre sexual likes and dislikes.
"it is OK for them to have bizarre sexual likes and dislikes."
I agree, so long as they are not consuming something that harmed anyone. Just as we know banning ivory sales reduces elephant poaching, contributing to demand causes harm. Putting an AI step in between the CP and its consumption is the same as before, it doesn't change anything.
It changes everything. It removes the part where harm needs to be done to satisfy demand. If someone says "I want a picture of X!" and an AI generates them the picture, no harm is done.
The harm was done. It was just done before the consumption step, just like buying an ivory shoehorn or making a photocopy of an existing image of child abuse.
The only way around it is the model is not trained on any child abuse material, otherwise legally it's consuming a CP product.
That’s what the state police charged him with in March. The article doesn’t include the specific federal charges for some reason, but the FBI statement suggest they believe the generation itself is also illegal.
> “Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material,” Principal Deputy Assistant Attorney General Nicole M. Argentieri said in an FBI press release.
The point of (d) is to create a jurisdictional hook for the US federal government. The federal government only has power over a limited number of offenses, including "interstate commerce" related offenses, but not most normal criminal offenses, which can only be criminalized by the states. So, for the federal government to regulate it, the easiest way is to only regulate things that involve interstate commerce somehow. In this case, downloading the Stable Diffusion model over the Internet probably creates enough of a hook, and this defendant is alleged to have done much more than that, so it's probably enough.
Interesting, so an obscene cartoon drawing in pencil with a pencil purchased locally would not be an offense, but if the pencil was purchased interstate or by mail then it is.
Are there any actual cases on this? i.e., someone downloads a model, creates but does not distribute the results, gets convicted?
I think prosecutors would probably argue that even using a pencil purchased locally meets the bar of making the "visual depiction... produced using materials.. or that have been shipped or transported in interstate or foreign commerce by any means, including by computer".
The Commerce Clause has been read very expansively since the Wickard v. Filburn[1] in the New Deal era. One of the current legal projects of the Federalist Society, the conservative legal movement the current Supreme Court majority stems from, is rolling this back, and limiting the power of the Commerce Clause. They argue that reading the Commerce Clause so expansively coupled with modern technological/economic change gives the Federal Government effectively unchecked power to regulate behavior, which was contrary to the intention/design of the Constitution. Supporters of the current status quo argue that reading the Commerce Clause too narrowly would make the modern economy unmanageable and ungovernable. It's an open question how far the current Supreme Court will go in paring it back, but they have started narrowing parts of this doctrine.[2]
(There are no prior cases on this in the context of AI generated content in the United States, because this is the first time this offense has been charged for AI generated content in the United States. That's why it's so newsworthy and why I posted it - it's going to set some precedent, one way or another.)
I personally find the much more interesting argument here to be about obscenity. The obscenity doctrine places "obscene" speech completely outside the protection of the First Amendment. Previously, the Supreme Court ruled in Ashcroft vs FSC that for CSAM to be illegal, it must at least meet the bar for obscenity, or it must be produced via actual exploitation.[3] This is the most similar case I am aware of.
Obscenity doctrine in the United States is... a bit of a hot mess, in my humble opinion. The current test is as follows:
> The basic guidelines for the trier of fact must be: (a) whether "the average person, applying contemporary community standards" would find that the work, taken as a whole, appeals to the prurient interest; (b) whether the work depicts or describes, in a patently offensive way, sexual conduct specifically defined by the applicable state law; and (c) whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value.
To be blunt, I don't think anyone really knows what that word salad means. What are the "contemporary community standards"? Who is the "community"? What does it mean to be "patently offensive"? According to whom? The whole thing that makes exclusions to the First Amendment tenable is that they need to be really, really clear. Otherwise ambiguity in the standard leads to curtailment of expression by a chilling effect.
Furthermore, there's also this ruling[4] - that as far as I can tell is still good law - which makes it unconstitutional to ban the mere possession of obscene material. I don't think that applies here, because in this case, the defendant is alleged to have distributed the material widely. Furthermore, it also stems from a "right to privacy" the current Supreme Court is rather skeptical of.
There's not really a conclusion to all this. The main point here is that something is happening and that will probably result in interesting decisions later on.
> I think prosecutors would probably argue that even using a pencil purchased locally meets the bar of making the "visual depiction... produced using materials.. or that have been shipped or transported in interstate or foreign commerce by any means, including by computer".
This would expand the Commerce Clause beyond Wickard v. Filburn arguably.
> Previously, the Supreme Court ruled in Ashcroft vs FSC that for CSAM to be illegal, it must at least meet the bar for obscenity, or it must be produced via actual exploitation.
You meant virtual child pornography? New York v. Ferber established Congress could ban real child pornography regardless of obscenity. I don't think Ashcroft v. Free Speech Coalition changed this. And calling all child pornography CSAM has made people confused about what images are illegal.
> The main point here is that something is happening and that will probably result in interesting decisions later on.
Possibly. Probably it will result in a plea bargain. And courts have resisted to clarify obscenity.
The first, second, and fourth charges in the indictment[1] is for violating 18 USC 1466A(a)(1)/(b)(1)/(d)(1)/(d)(4), which are the statutes against producing/distributing/possessing an "obscene" "drawing, cartoon, sculpture, or painting" "depict[ing] a minor engaging in sexually explicit conduct".
Pedophilia is a paraphilia. Most psychologists believe paraphilias are permanent. Therapy and medication can help people control their urges if needed. Child sex abuse is a criminal choice.
Yeah, I had to work with SOs in my previous job and this sounds accurate. It's very hard to characterize this kind of thing though so if anyone else is curious it would be best to read up a bit on modern approaches.
As of now there is no proof that it can be cured. Most treatments and therapies are to help pedophiles control their urges and not act.
At the end of the day, it's a paraphilia. I don't think you can cure someone who's aroused by feet or uniforms. I don't believe there is any evidence today that you can alter people's sexual interests.
I guess there are two cases, a Wisconsin state case (March, out on bail, still pending) and this federal case. Apparently it is not double jeopardy to re-arrest for the same thing (or different statutes, in this case) if one is state and one federal.
With an image of a real scene, I suppose it would be a defense against a child porn charge to prove that the participants were not minors. But there could be no such defense if the porn is generated. It becomes a matter for judges and juries to decide if a given picture of a person is over or under a particular age. I wonder if it would be a defense to prove that the prompt did not designate an illegal age.
It may be a useful liability hedge to provably include the prompt in generated content, like as a watermark. But since existing generation isn't repeatable, and the prompt can't reliably generate the same image, is that even possible?
"Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal"
Not with the charges given. Prosecutors seem keen to push the normalisation angle, but would then have to explain why FPS games aren't also illegal. Unless they can prove real CSAM was used to train the model so they can draw a line from the material to the defendant's production.
37 comments
[ 4.7 ms ] story [ 1687 ms ] thread>“Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material"
That "or distribute" should really be "and distribute"; the whole point should be to protect kids, not to punish people that are indeed fucked up but found a way to not harm the society.
It is a total waste of taxpayer funds to be going after someone if all they've done is generate however many images with stable diffusion. I'm glad in this case he's at least done something objectionable although an arrest seems a bit of an overreach, this seems to be the sort of situation that could be resolved with a fine rather than jail time based on skimming the indictment. If that.
But the harm in it comes from generating further demand and normalising the sexualization of children. And than can happen with any material from before or after 1850. If a model is trained on it, it becomes a derivative work that should be treated no differently than a b&w version of a colour abuse image.
Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service.
Sounds like I'd be happy to go with that. You're taking it to the logical conclusion and I generally agree. The problem to me is (1) people abusing children and (2) people paying other people to abuse children.
> Not sure what tax dollars have to do with this but IMO it's a small price to pay for an important service.
It isn't an important service. With the advent of generative AI models, going after people for possessing dodgy images is now more than likely persecuting people who haven't done any harm to anyone and aren't going to.
We really should have learned this lesson after Turing. If people aren't harming anyone then it is OK for them to have bizarre sexual likes and dislikes.
I agree, so long as they are not consuming something that harmed anyone. Just as we know banning ivory sales reduces elephant poaching, contributing to demand causes harm. Putting an AI step in between the CP and its consumption is the same as before, it doesn't change anything.
The only way around it is the model is not trained on any child abuse material, otherwise legally it's consuming a CP product.
> “Today’s announcement sends a clear message: using AI to produce sexually explicit depictions of children is illegal, and the Justice Department will not hesitate to hold accountable those who possess, produce, or distribute AI-generated child sexual abuse material,” Principal Deputy Assistant Attorney General Nicole M. Argentieri said in an FBI press release.
[1] https://www.justice.gov/opa/pr/man-arrested-producing-distri...
[2] https://www.law.cornell.edu/uscode/text/18/1466A
The point of (d) is to create a jurisdictional hook for the US federal government. The federal government only has power over a limited number of offenses, including "interstate commerce" related offenses, but not most normal criminal offenses, which can only be criminalized by the states. So, for the federal government to regulate it, the easiest way is to only regulate things that involve interstate commerce somehow. In this case, downloading the Stable Diffusion model over the Internet probably creates enough of a hook, and this defendant is alleged to have done much more than that, so it's probably enough.
Are there any actual cases on this? i.e., someone downloads a model, creates but does not distribute the results, gets convicted?
The Commerce Clause has been read very expansively since the Wickard v. Filburn[1] in the New Deal era. One of the current legal projects of the Federalist Society, the conservative legal movement the current Supreme Court majority stems from, is rolling this back, and limiting the power of the Commerce Clause. They argue that reading the Commerce Clause so expansively coupled with modern technological/economic change gives the Federal Government effectively unchecked power to regulate behavior, which was contrary to the intention/design of the Constitution. Supporters of the current status quo argue that reading the Commerce Clause too narrowly would make the modern economy unmanageable and ungovernable. It's an open question how far the current Supreme Court will go in paring it back, but they have started narrowing parts of this doctrine.[2]
(There are no prior cases on this in the context of AI generated content in the United States, because this is the first time this offense has been charged for AI generated content in the United States. That's why it's so newsworthy and why I posted it - it's going to set some precedent, one way or another.)
I personally find the much more interesting argument here to be about obscenity. The obscenity doctrine places "obscene" speech completely outside the protection of the First Amendment. Previously, the Supreme Court ruled in Ashcroft vs FSC that for CSAM to be illegal, it must at least meet the bar for obscenity, or it must be produced via actual exploitation.[3] This is the most similar case I am aware of.
Obscenity doctrine in the United States is... a bit of a hot mess, in my humble opinion. The current test is as follows:
> The basic guidelines for the trier of fact must be: (a) whether "the average person, applying contemporary community standards" would find that the work, taken as a whole, appeals to the prurient interest; (b) whether the work depicts or describes, in a patently offensive way, sexual conduct specifically defined by the applicable state law; and (c) whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value.
To be blunt, I don't think anyone really knows what that word salad means. What are the "contemporary community standards"? Who is the "community"? What does it mean to be "patently offensive"? According to whom? The whole thing that makes exclusions to the First Amendment tenable is that they need to be really, really clear. Otherwise ambiguity in the standard leads to curtailment of expression by a chilling effect.
Furthermore, there's also this ruling[4] - that as far as I can tell is still good law - which makes it unconstitutional to ban the mere possession of obscene material. I don't think that applies here, because in this case, the defendant is alleged to have distributed the material widely. Furthermore, it also stems from a "right to privacy" the current Supreme Court is rather skeptical of.
There's not really a conclusion to all this. The main point here is that something is happening and that will probably result in interesting decisions later on.
[1] https://fedsoc.org/case/wickard-v-filburn; held that Congress could bar farmers from growing wheat on their own land for their own consumption. [2] https://www.nlc.org/article/2023/06/15/supreme-court-decides... limited the rel...
This would expand the Commerce Clause beyond Wickard v. Filburn arguably.
> Previously, the Supreme Court ruled in Ashcroft vs FSC that for CSAM to be illegal, it must at least meet the bar for obscenity, or it must be produced via actual exploitation.
You meant virtual child pornography? New York v. Ferber established Congress could ban real child pornography regardless of obscenity. I don't think Ashcroft v. Free Speech Coalition changed this. And calling all child pornography CSAM has made people confused about what images are illegal.
> The main point here is that something is happening and that will probably result in interesting decisions later on.
Possibly. Probably it will result in a plea bargain. And courts have resisted to clarify obscenity.
A prosecutor would say yes. Would a court?
https://en.m.wikipedia.org/wiki/Thoughtcrime
There is 90s US precedence and then laws were changed to make “obscene” content illegal
https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343
https://www.theverge.com/2014/8/19/6044679/the-six-lawsuits-...
Here is one 1997 case when snuff porn on alt.sex.stories passed Freedom of Speech test and the defendant was found not guilty
https://en.m.wikipedia.org/wiki/United_States_v._Alkhabaz
[1] https://www.justice.gov/opa/media/1352606/dl?inline
At the end of the day, it's a paraphilia. I don't think you can cure someone who's aroused by feet or uniforms. I don't believe there is any evidence today that you can alter people's sexual interests.
It may be a useful liability hedge to provably include the prompt in generated content, like as a watermark. But since existing generation isn't repeatable, and the prompt can't reliably generate the same image, is that even possible?
i'm not really educated on what treatment looks like for these people, and harms to society from AI CSAM should obviously be considered too.
but some kind of "prescription" or otherwise regulated generation of these images should at least be considered, i think.
Not with the charges given. Prosecutors seem keen to push the normalisation angle, but would then have to explain why FPS games aren't also illegal. Unless they can prove real CSAM was used to train the model so they can draw a line from the material to the defendant's production.
Let's see if there are more charges to come.