The court was asked to annul the previous judgment since the defendant did not think that the pictures in question were CSAM, since the children depicted were fictional. The court found the pictures similar to photography and therefore rejected the appeal.
In my mind, this is pretty substantial technology news, at least in Denmark, since it rejects the notion that AI generated pictures are substantially different from other pictures, and therefore has to abide by the same laws.
I will be interested in seeing if anybody charges the big AI providers for producing these photos.
Shit like this is why I can't take any of the "freedom indexes" seriously. Europe is constantly ranked highly but then you get shit like oh you're fake picture looks a little too real jail.
Maybe you're just not considering the nuance of the situation. You don't have to take them seriously, but you also don't have to minimize it by misrepresenting words no one said.
Yeah those guys are so not free because they ban CP, a thing you think should be allowed as long as it's not real. Do you see how easy it is to make a bad argument?
CSAM laws exist to punish abusers, and seek justice for the abused. Not one or the other, but both. The law cannot function without the victims, else you're just attacking people for what amounts to thought crimes.
I find CSAM and pedophiles disgusting, and I imagine 100% of HN users agree with me on this, but this just opens the door for world governments to a) claim LLM output is real and factual (!), but b) minimizes the danger to victims and equates victimhood to being a fictional character.
I'm not Dutch and I'm hoping this is just some Dutch-specific weird quirk of their law, but from what I can see, this is a politically-biased ruling to "think of the children".
Should we allow victims of child pornography to endorse the distribution of images of themselves as a child once they are an adult too since at that point, each additional viewer doesn't generate an additional victim? No, because it encouraged future abuse.
Unfortunately that gets into a weird spot. I, too, believe that it should be disallowed for that reason, but the opposite end of that is prosecuting teenagers for distributing photos of themselves to other teenagers (their same-age peers)... which has unfortunately occurred due to improper application of the law.
The law must recognize the right to self-determinism and bodily autonomy, and you can't inherently be both the victim and the abuser. There is no one size fits all law, and many governments are trying to act like they have one.
> ... but the opposite end of that is prosecuting teenagers for distributing photos of themselves to other teenagers (their same-age peers)... which has unfortunately occurred due to improper application of the law.
Do you have a concrete example of this happening? Every time I have heard of a minor being persecuted for possession or distribution is when one person sent an image of themselves to another person and then that other person further distributed it to other people without their consent, which to be clear is wrong.
That is part of the problem though: in any acceptable interpretation of the law, the one in the photo would be the victim, so how could they be charged with also being the abuser? You can't be your own abuser. I, too, am only aware of cases where this has happened because the receiver then shared it with others; however, the person that received it is also a minor of the same age thus also inappropriate to charge them, but would be the more correct choice.
> however, the person that received it is also a minor of the same age thus also inappropriate to charge them
No not really, sharing sexual images to others that were shared to you directly by a person should be (and in some countries is) a crime. The boy in the linked case should be the one charged with the distribution of child pornography, or something akin to revenge porn since they are both minors.
It is probably very hard to carve out these exceptions for minors sexting, but we already have (in some countries) consent laws that factor in relationships between minors. I don't see why this would be different
It depends on the culture. Many crimes like this are considered adult crimes, and to be charged with them as a minor the prosecution has to go through the process to get them charged as an adult.
Generally the process isn't worth it unless they think they have a near certain chance of winning against the child.
I do agree, however, that in bird culture, sharing nudes that were sent to you is considered a dick move.
I agree because I think it is a slippery slop, is that the same reasoning can soon easily be applied to other more disputable cases.
Look at the recent news that Anthropic will ban users that insult their models. How can it make any logic sense?
But it could easily become "punishable by law" also:
- Abusing (non sexually) a fictitious person that is a LLM.
- Insulting a LLM in a chat box, based on a fact that it speaks and appear to behave like a human. The LLM can pretend and simulate that it is offended if you forget that it is just bytes behind.
And in the end, how to consider the following case:
- You have a government call center for a public service, you are aggressive and insult the agent -> it is potentially a legal crime
- Now the agent of this public service that you are talking to is in fact an AI, clearly stated as so, an you insult it very aggressively. -> No human being is hurt so should it be an legal offense?
There's a few comments (boldly) criticizing the ruling for being technically incorrect. If no child was abused, how can it be a child abuse crime?
I would counter with this: If it is technically impossible to tell the difference between AI generated images, and real ones, how could anyone ever be prosecuted for having presumptive real ones in their possession ever again? Everyone tried would simply say "AI" and that'd be that, unless there was a victim to testify, or it was a case on generation of it etc.
So AI images of this nature do have victims, just slightly less directly.
Another adjacent question: How would you feel if it was an AI generated image of a specific child. What is the precise threshold of victimhood that is necessary?
I disagree. If it's impossible to tell the difference, people WILL start distributing collections of real and artificial child pornography combined while labeling it as 100% artificial. Most public websites will make both against the terms of service, but it makes it much harder for victims to get support from law enforcement to take down genuine CSAM.
Also, now that it's legal to own and consume content featuring realistic depictions of sexualized children, do you think that will lead to fewer or more children getting abused?
I wonder how people (like yourself) would would feel about "un-watermarked AI-generated CSAM is illegal, but watermarked is not". Or is is your first argument irrelevant and the second one is what actually matters to you?
> Also, now that it's legal to own and consume content featuring realistic depictions of sexualized children, do you think that will lead to fewer or more children getting abused?
I'm not sure. Is the intuition that it will lead to more? because I think there are data-based arguments to the contrary.
I'm not convinced you can stop someone from taking real CSAM, and recreating it with AI to get a copy with a watermark.
Also, assuming your correct and someday we have some watermarked attested "victimless" child pornography. I'd love to hear your best "data-based" argument that shows that predators will be satiated by pictures they have no doubt about being fake. It's laughable.
Should we allow victims of child pornography to endorse the distribution of images of themselves as a child once they are an adult too since at that point, each additional viewer doesn't generate an additional victim?
You're just making more criminals this way. It's the same as saying air rifles that are indistinguishable from fully automatic weapons should be treated as fully automatic weapons.
I hate to extend a bad analogy, but If you rob a convenience store with an airsoft rifle that is visually indistinguishable from a real gun, you will be charged with armed robbery.
People are already using AI to generate porn of real minors, so you don't have to get hypothetical to see how the technology is used to abuse children. If it's easy for any predator to do this, then it's not hard to imagine that they will blackmail minors by threatening to release AI nudes to their friends and family unless they send them real CSAM.
You've combined two crimes into one. Robbing a bank with anything is armed robbery. My analogy is apt.
If I get pulled over with an airsoft identical to an AK47 and charged with trafficking illegal firearms, I think you will agree that I shouldn't be. It's not an AK47, it's an airsoft.
Similarly, calling AI generated imagery CSAM might be a pure category error. There is no C being SA, so it's not CSAM by definition.
Now if that imagery came about because real CSAM material was in the training data, that's a different story. You now have a derivative work based on a real crime with real victims. But we'd also have some very real and important questions to be asking the model trainers.
In a vacuum though? Sorry no, it doesn't make any sense.
Generating sexualized pictures of minors and normalizing pedophillia is soliciting more abuse. You are willfully ignore that at this point to defend your shitty analogy. Your analogy sucks because child porn, and especially it's distribution is inherently bad for society. Trafficking an airsoft gun does nothing. Your semantic arguments about the acronym are extremely cliche especially since you know exactly what everyone means. Before some people started calling it CSAM, it was called child pornography AND IT WAS STILL ILLEGAL
I think you're being overly emotional and haven't addressed any of my points. You just doubled down and said it's child porn again when I've stated to you a couple times that if the porn contains no child, it can't be child porn by definition.
And to be clear I'm not a consumer of such things nor am I attempting to normalize anything. I just want to find out the truth about stuff.
> I would counter with this: If it is technically impossible to tell the difference between AI generated images, and real ones, how could anyone ever be prosecuted for having presumptive real ones in their possession ever again?
If AI CSAM is legal, the market for real CSAM is effectively destroyed. It's heartbreaking to me that--incredibly--we're making our best weapon in the fight against child abuse illegal. It's the methadone of CSAM. What the fuck are we doing?
If you cannot tell the difference, you cannot tell the difference. Unfortunate as it may be, the standard of proof for criminal conviction must be high. There's a reason that various legal systems across the world have converged on such principles as proof beyond reasonable doubt, in dubio pro reo, intime conviction etc. basically stating that the legal burden of proof is on the prosecution to present compelling evidence of crime and that the benefit of doubt must go to the accused. It is considered so fundamental and so important that it has made it into the International Covenant on Civil and Political Rights which a majority of countries in the world (including Denmark) have ratified. There must no doubt have been painful lessons in history which led humanity to eventually adopt such ideas. We must be careful to forget those and inadvertently take steps going back towards that direction.
This doesn't make sense. There's law enforcement and privately run databases you can check hashes of known material from.
The argument that producing this material caused harm always made sense but when it came to stories and loli anime drawings, it never made sense.
Back in the 2010s when Germany started censoring DNS with csam sites (of which 99% were false listings) I attended a protest and got to talk to a policeman who talked about the methods they use to catch producers by looking at background sounds, unique identifiers on both perpetrator and victim (tattoos, scars, etc), where light came from, where shadows landed, visible products in the background that might help triangulate possible locations or narrow down to country/city/etc.
I don't see why police wouldn't have to keep doing their job to get to hunt down predators.
Making generated content outright illegal doesn't align with my understanding of free speech.
I'm seeing some other comments talking about how no one is harmed by AI CSAM. Fundamentally, I reckon AI CSAM is the same as Lolicon, which is illegal here in Sweden. I believe the same applies to Denmark, though I am by no means knowledged about Danish law.
AI is not something new in this space, it just means that a lot more grotesque fictional CSAM can be produced. It has been possible to create realistic-looking-but-fictional CSAM since Photoshop came out (and then some).
> I reckon AI CSAM is the same as Lolicon, which is illegal here in Sweden. I believe the same applies to Denmark, though I am by no means knowledged about Danish law.
This interests me. I don't know of any specific ruling we can draw any conclusion from, but the ruling in question here cites some procedural texts that specifically state that the danish CSAM laws only cover "reproductions", which they then also specifically define as not including "drawing and paintings, such as antique vases".
It seems to me that a compelling argument could be made on both sides of that distinction. On the one hand, lolicon shared on X share nothing in common with art on antique vases. On the other, they would not be included in the textual definition of reproduction.
In this specific matter I think it's most likely that drawings of CSAM is legal, at least under these statutes.
Outlawing AI CSAM is basically outlawing people from having certain types of sexual thoughts and feeling. In the case of materials that produce pedophilic sexual thoughts and feelings, I think it is legitimate to outlaw them.
Certain materials are designed to produce sexual feelings. And I am not saying that having the thoughts is or should be a crime, though I think that having them is morally wrong.
Outlawing materials that are designed to produce pedophilic thoughts and consistently do so for some people is a way for society to say that such thoughts are wrong. I think part of being a responsible human being is controlling ones own thoughts to some ways.
Some materials aren't designed thus yet produce sexual feelings anyway. Some materials produced to generate sexual feelings produce disgust instead. This reasoning is why we have always seriously questioned the banning of anything.
> The expression became one of the best-known phrases in the history of the Supreme Court.[4] Though "I know it when I see it" is widely cited as Stewart's test for "obscenity", he did not use the word "obscenity" himself in his short concurrence, but stated that he knew what fit the "shorthand description" of "hard-core pornography" when he saw it.
Someone who produces materials that produce sexual feelings in a large proportion of people but didn't realize that would happen is an idiot. There is no way to protect idiots from negative consequences for their actions.
So your saying that people who use ai to generate videos of children having sex with adults don't realize it would sexually stimulate a certain class of viewers?
I'm saying what's arousing is subjective. No one tries to ban videos of someone getting beans poured on their head because it's consensual and there is no one getting hurt. But it is weird. An AI image of a naked child is not an image of a child (assuming no children in training data etc), and there is no victim. Some people are aroused by this but most people aren't. Why should it be banned? Make an argument.
"is subjective" if you are saying that it varies from person to person, I agree. What is am saying is that anyone with an ounce of intelligence can predict that certain materials will produce sexual feelings in a certain existing class of people, namely pedophiles. The fact that you have not directly denied this indicates that you know I am right but don't want to admit it.
As far as an argument for why this is bad, I produced it in my first comment, and you have not responded to it.
I think that we have both made our positions clear to anyone who has been reading this comment thread, so I am not going to reply to any future comments you might make, and I will just leave it up to people to dedide for themselves which of us is right.
Of course sexual materials of all kinds will stimulate some people. This isn't an argument though, it's just an observation. Pedophiles often have no access to materials but get turned on by their own fantasies. Obviously all abuse material should be illegal, not because it stimulates feelings but because there are actual human victims involved. But with AI generated material, there isn't a victim (theoretically) so why should it be banned? Because it satisfies pedophiles? Or are you making an argument about adding fuel to a fire here- like if they watch AI porn they're more likely to go commit real crimes or get more deep into their twisted desires? Because that would be an argument, and one that I think has merit.
It's slightly misleading to refer to 'CSAM' in the title, as the article doesn't actually use it.
CSAM is an American legal concept, and the idea that child porn should be legal as long as there's no provable abuse is a peculiarly American take. Most European countries don't work like that.
Danish law appears to outlaw (roughly translated) "sexual material containing visual representations of persons under the age of 18". Whether or not a specific minor was abused doesn't come into it. So clarifying that that covers AI-generated imagery seems to be a much smaller step than many in this thread are assuming.
57 comments
[ 0.16 ms ] story [ 11.9 ms ] threadIn my mind, this is pretty substantial technology news, at least in Denmark, since it rejects the notion that AI generated pictures are substantially different from other pictures, and therefore has to abide by the same laws.
I will be interested in seeing if anybody charges the big AI providers for producing these photos.
Yeah those guys are so not free because they ban CP, a thing you think should be allowed as long as it's not real. Do you see how easy it is to make a bad argument?
Nope, I just see someone making bullshit excuses.
Also your account is 10 hours old, hows the Russian datacenters this time of year?
CSAM laws exist to punish abusers, and seek justice for the abused. Not one or the other, but both. The law cannot function without the victims, else you're just attacking people for what amounts to thought crimes.
I find CSAM and pedophiles disgusting, and I imagine 100% of HN users agree with me on this, but this just opens the door for world governments to a) claim LLM output is real and factual (!), but b) minimizes the danger to victims and equates victimhood to being a fictional character.
I'm not Dutch and I'm hoping this is just some Dutch-specific weird quirk of their law, but from what I can see, this is a politically-biased ruling to "think of the children".
The law must recognize the right to self-determinism and bodily autonomy, and you can't inherently be both the victim and the abuser. There is no one size fits all law, and many governments are trying to act like they have one.
Do you have a concrete example of this happening? Every time I have heard of a minor being persecuted for possession or distribution is when one person sent an image of themselves to another person and then that other person further distributed it to other people without their consent, which to be clear is wrong.
https://www.aclu.org/news/juvenile-justice/minnesota-prosecu...
This is one such case, but hardly unique.
No not really, sharing sexual images to others that were shared to you directly by a person should be (and in some countries is) a crime. The boy in the linked case should be the one charged with the distribution of child pornography, or something akin to revenge porn since they are both minors.
It is probably very hard to carve out these exceptions for minors sexting, but we already have (in some countries) consent laws that factor in relationships between minors. I don't see why this would be different
Generally the process isn't worth it unless they think they have a near certain chance of winning against the child.
I do agree, however, that in bird culture, sharing nudes that were sent to you is considered a dick move.
Look at the recent news that Anthropic will ban users that insult their models. How can it make any logic sense?
But it could easily become "punishable by law" also:
- Abusing (non sexually) a fictitious person that is a LLM.
- Insulting a LLM in a chat box, based on a fact that it speaks and appear to behave like a human. The LLM can pretend and simulate that it is offended if you forget that it is just bytes behind.
And in the end, how to consider the following case:
- You have a government call center for a public service, you are aggressive and insult the agent -> it is potentially a legal crime
- Now the agent of this public service that you are talking to is in fact an AI, clearly stated as so, an you insult it very aggressively. -> No human being is hurt so should it be an legal offense?
I would counter with this: If it is technically impossible to tell the difference between AI generated images, and real ones, how could anyone ever be prosecuted for having presumptive real ones in their possession ever again? Everyone tried would simply say "AI" and that'd be that, unless there was a victim to testify, or it was a case on generation of it etc.
So AI images of this nature do have victims, just slightly less directly.
Another adjacent question: How would you feel if it was an AI generated image of a specific child. What is the precise threshold of victimhood that is necessary?
Also, now that it's legal to own and consume content featuring realistic depictions of sexualized children, do you think that will lead to fewer or more children getting abused?
> Also, now that it's legal to own and consume content featuring realistic depictions of sexualized children, do you think that will lead to fewer or more children getting abused?
I'm not sure. Is the intuition that it will lead to more? because I think there are data-based arguments to the contrary.
Should we allow victims of child pornography to endorse the distribution of images of themselves as a child once they are an adult too since at that point, each additional viewer doesn't generate an additional victim?
No?
People are already using AI to generate porn of real minors, so you don't have to get hypothetical to see how the technology is used to abuse children. If it's easy for any predator to do this, then it's not hard to imagine that they will blackmail minors by threatening to release AI nudes to their friends and family unless they send them real CSAM.
If you generate defamatory AI pics of actual people, there are already laws against that, and the CSAM aspect can be exacerbating.
Otherwise, you should be free to jerk it to whatever your heart desires, which seems to be the real crux of the matter.
If I get pulled over with an airsoft identical to an AK47 and charged with trafficking illegal firearms, I think you will agree that I shouldn't be. It's not an AK47, it's an airsoft.
Similarly, calling AI generated imagery CSAM might be a pure category error. There is no C being SA, so it's not CSAM by definition.
Now if that imagery came about because real CSAM material was in the training data, that's a different story. You now have a derivative work based on a real crime with real victims. But we'd also have some very real and important questions to be asking the model trainers.
In a vacuum though? Sorry no, it doesn't make any sense.
And to be clear I'm not a consumer of such things nor am I attempting to normalize anything. I just want to find out the truth about stuff.
Like once the cop finds out the gun isn't real, he retracts the charges for arms trafficking. Right?
If AI CSAM is legal, the market for real CSAM is effectively destroyed. It's heartbreaking to me that--incredibly--we're making our best weapon in the fight against child abuse illegal. It's the methadone of CSAM. What the fuck are we doing?
(Side note, talk to someone whose done methadone sometime, it's not as magical as it sounds)
The argument that producing this material caused harm always made sense but when it came to stories and loli anime drawings, it never made sense.
Back in the 2010s when Germany started censoring DNS with csam sites (of which 99% were false listings) I attended a protest and got to talk to a policeman who talked about the methods they use to catch producers by looking at background sounds, unique identifiers on both perpetrator and victim (tattoos, scars, etc), where light came from, where shadows landed, visible products in the background that might help triangulate possible locations or narrow down to country/city/etc.
I don't see why police wouldn't have to keep doing their job to get to hunt down predators.
Making generated content outright illegal doesn't align with my understanding of free speech.
AI is not something new in this space, it just means that a lot more grotesque fictional CSAM can be produced. It has been possible to create realistic-looking-but-fictional CSAM since Photoshop came out (and then some).
This interests me. I don't know of any specific ruling we can draw any conclusion from, but the ruling in question here cites some procedural texts that specifically state that the danish CSAM laws only cover "reproductions", which they then also specifically define as not including "drawing and paintings, such as antique vases".
It seems to me that a compelling argument could be made on both sides of that distinction. On the one hand, lolicon shared on X share nothing in common with art on antique vases. On the other, they would not be included in the textual definition of reproduction.
In this specific matter I think it's most likely that drawings of CSAM is legal, at least under these statutes.
Outlawing materials that are designed to produce pedophilic thoughts and consistently do so for some people is a way for society to say that such thoughts are wrong. I think part of being a responsible human being is controlling ones own thoughts to some ways.
> The expression became one of the best-known phrases in the history of the Supreme Court.[4] Though "I know it when I see it" is widely cited as Stewart's test for "obscenity", he did not use the word "obscenity" himself in his short concurrence, but stated that he knew what fit the "shorthand description" of "hard-core pornography" when he saw it.
As far as an argument for why this is bad, I produced it in my first comment, and you have not responded to it.
I think that we have both made our positions clear to anyone who has been reading this comment thread, so I am not going to reply to any future comments you might make, and I will just leave it up to people to dedide for themselves which of us is right.
CSAM is an American legal concept, and the idea that child porn should be legal as long as there's no provable abuse is a peculiarly American take. Most European countries don't work like that.
Danish law appears to outlaw (roughly translated) "sexual material containing visual representations of persons under the age of 18". Whether or not a specific minor was abused doesn't come into it. So clarifying that that covers AI-generated imagery seems to be a much smaller step than many in this thread are assuming.