More details on gov.uk [1] but still without a definition of what a 'deepfake' (their quotes) actually is. Reference is made to the law on child pornography, for that no computer (nor child) need be involved. A crude pencil drawing falls within the remit of the law, so I guess something along those lines ...
I hope they're very careful with the eventual definition, as at this point you might call Photoshop's AI-driven outpainting feature a "deepfake." If an image is already explicit, and you use AI to de-crop a missing foot back into the picture, have you now created an "explicit deepfake"?
> Laura Farris, the minister for victims and safeguarding, said the creation of deepfake sexual images was “unacceptable irrespective of whether the image is shared”.
This is an interesting one because it violates the vague principle of "whatever done by consenting adults in the privacy of their own home is cool".
Like what if someone pastes my face ona naked body, or imagines me naked, or has a dream about me flying through the window naked and writes it in their private dream diary.
I think it's getting into hairy legal territory if the "infringing material" is only seen by the otherwise innocent creator on an airgapped computer.
The existence of a CSAM implies a crime was committed though, since there is no way to legally produce such an image. Possessing a CSAM image is pretty much the same as aiding/abetting the original crime.
That's not true, the justification outlawing CSAM being the result of a crime is not the actual legal justification for banning it and it would not at all hold up in court.
There are plenty of incredibly disturbing videos and images of some of the most gruesome crimes committed and they are lawfully accessible and obtainable on the open Internet, including some such images on Wikipedia. The documenting of a crime is not and has never been illegal.
You might be right, based on Kranar's comment. I thought that CSAM was illegal to artificially create, as well, which would be similar to a deepfake.
On a related note, I assume there is already a lot of deepfake CSAM in certain circles. If that is ruled legal, it will make finding and prosecuting real life CSAM creators much more difficult.
Depends. Some jurisdictions classify CSAM by end results, others by process. Where it's process oriented it's not CSAM unless a child is not physically abused.
> The existence of a CSAM implies a crime was committed though, since there is no way to legally produce such an image
Well if those images are illegal, you're right that there's no way to produce them legally, but that's because they are on their face illegal.
If you mean there is no way to produce CSAM that doesn't break other laws, I think you're wrong - e.g. see the case of teenagers taking nude pictures of themselves. That is an otherwise legal thing to do, but having those images is illegal, as I understand it, and you can search Google for many examples of teens prosecuted for sexting images of themselves. I don't know if anyone has been prosecuted for merely having nude images of themselves, but that might just be because no one would even know about it, necessarily.
You are certainly right and there is good reason for that, however, it's worth noting that in the U.S., at least as per Ashcroft v. Free Speech Coalition [1], simulated CSAM is legal.
No, I was careful to not make any absolutist claim and even provided a reference for people to read further. Pornography in general is subject to the Miller test, so it should not be surprising that simulated CSAM is also subject to the Miller test as well.
"simulated CSAM" is a strange term - "CSAM" itself could be read as a dysphemism of CP whose purpose is less to emphasize the heinous nature of the abuse involved, and more to defend "ordinary" pornography as licit by putting a semantic distance between it and child porn. But of course if it is simulated, there is no abuse going on so the term becomes meaningless.
> "CSAM" itself could be read as a dysphemism of CP whose purpose is less to emphasize the heinous nature of the abuse involved
Maybe it could be read that way by some, but that's not why the term was adopted and it's not why the term is advocated for by anti-sexual violence organizations.
As for the fact that making it simulated renders the term meaningless, the entire point of a simulation is that it is not the actual thing that is happening but rather an imitative representation of something that could be real. Presumably the abuse towards a child could be real, and hence it's possible to imitate that abuse in various forms of media. It is that imitation that constitutes the simulation.
If you argue that it makes the term meaningless, then you must accept that the simulation of anything is meaningless.
> If you argue that it makes the term meaningless, then you must accept that the simulation of anything is meaningless.
I think that what is being argued. Put 'simulated' in front of any legal term and it becomes meaningless for its legal usage. 'Simulated larceny' is meaningless. 'Simulated murder' is meaningless.
Yeah, good luck simulating a terrorist attack at an airport, or simulating legal tender, or simulating a police officer.
There are plenty of laws that make it an offense to perform a simulation. Nitpicking the term simulation as being meaningless, whether in a legal context or otherwise, isn't a particularly noteworthy discussion to have. I think most people understand the meaning of simulated CSAM and can have an opinion on whether it should be legal or illegal without being pedantic about the word "simulated".
CSAM means something specific. It means 'a child was abused, here is evidence'. A simulation of that would be training materials for LEA or forensics teams to practice their craft.
You are really stretching here, and it is not 'pedantic' to ask that specific terms have specific meanings especially when it is a legal issue.
>A simulation of that would be training materials for LEA or forensics teams to practice their craft.
You are laughably misinformed on this matter in a way that is quite incomprehensible. You are welcome to refer to the legal definition as per the actual legal text instead of continuing to make rather bizarre claims on a topic where it's clear you know nothing about:
Even without having to refer to the actual text, I am confident most reasonable minded people understand the meaning of the term "simulated" and can take a position on whether it should be legal or not without arguing whether simulation means training materials or whether the term is legally meaningless.
The fact that you don't is all I need to know that continuing this conversation with you is entirely pointless.
I think the issue here is the between consenting adults part, deepfakes in this instance are probably not between consenting adults. I also think it's important to think about the reality of how this likely plays out. If someone deepfakes me and faps and I never find out and they never tell anyone, it would be quite hard for there to be action. So if someone is deepfaking you and you know about it, there must be a logical path there, and none of them look good for the deepfaker. Can't say I know where I land on the issue, and I've thought about it for days and hours.
Except the actual human being who resembles the deepfake image isn't actually involved at all.
People are acting like a "deepfake of a person" is something different than a drawing, painting, song, poem or any other artistic creation inspired by, based on, or with elements drawn from a real person.
"If someone is deepfaking you" is such a weird imaginary problem. Someone might be imagining you. Someone might see you in a coffee shop while writing a screenplay and use the outfit you're wearing to describe a character in their story. Who cares?
If someone is committing fraud by trying to pass off a deepfake as a real photo, then we already have laws around fraud, defamation, harassment to cover that. If someone makes a deepfake that looks similar to a real person for their own amusement, that's no different than writing fan-fiction or sketching in a notebook.
You don't have a right to consent to being part of an artwork. For example, even the King of England could not walk into a museum and demand they take a painting down because it's unflattering.
In fact, if you're not old enough, it's just flat out illegal. You'll go to prison for making that artwork. And you arguing that it's "art" is not going to find much resonance in US courts at least. You're likely to fail the Miller test if US citizens are the judges.
For sure but for the otherwise innocent creator on an airgapped pc there is nothing to worry about. Who would ever know?
I do think the other examples you gave have one significant and fundamental difference - no one would mistake those things as being real. The problem with deepfakes is these have a similar social impact and capcity for harm as real nudes do, only there is no element of consent. You could ruin someones life quite easily with deep fakes, but not with your lewd poetry or sketches. So it seems reasonable to me to ban these with a blanket ban.
I would prefer we lived in a world where that wasnt neccesary, but for every technology you must imagine the most sinister and exploitative application and realise that someone will try that and worse. How do you even begin to combat deepfakes and the social problems they bring? how many bodies are considered acceptable until we get it right? Possibly, its best to over reach a little then relax the grip rather than to take it slow, given what is on the line for some.
and again, if they will never be shared, there is not much to worry about. We do need tools to persecute people who failed on the 'never be shared' front. Be it theft or otherwise, if others get the material from them, it doesn't matter, it should be treated as guilty of producing with intent to distribute. This is for two reasons, the first being this way people cannot hide behind the 'oopsie' defense.Second, if you made compromising material of others without consent, your duty of care is through the roof. If it leaks, you are at fault, no matter how hard you tried to prevent it. You never had to make that material. There was no reason to make it other than a selfish self-centered personal pursuits of a hedonistic nature, now if its victimless I don't care personally but the moment your selfish self-centered behavior hurts others, you are a danger to society.
Don't you think that's impractical? All attempts to stop people from using the Internet to do things on their own have failed. If the creation of such images is considered a crime in itself, then it will be obscured. Therefore, treating dissemination as a crime is not only logical, but also practical.
And when it comes to accidental leaks, even if the person who created the image didn't pay attention, the leak happens by somebody anyway. I think it would be more practical and lead a better result to punish those responsible for the leak itself
>You could ruin someones life quite easily with deep fakes, but not with your lewd poetry or sketches. So it seems reasonable to me to ban these with a blanket ban.
A ban won't stop bad people from making them anyway. On the other hand, proliferation of deepfakes would make the existence of certain real or fake photos insignificant, as everyone would accept that anyone could make them easily.
Because then people can and will create fine tunes of models trained on specific people and just share those instead. I'm worried we will soon hear about a girl bullied into suicide because of a model she was powerless to stop from spreading.
The implication is that only one party has consented when these fakes are created, and therefore not all involved have given their consent.
Courts have previously deemed cartoon imagery of children to be equivalent to a real child, so considering the deepfake to be a real person that needs to give consent is not without precedent.
The basic obvious issue is “can you draw porn of someone”.
The implied answer here appears to be “no” and the reason this is deemed necessary is an emergent effect of scale of being able to do this at such volume and quality.
Saying you can’t draw porn of people without regard to implied authenticity or medium seems somehow more reasonable to me.
There just seems to be so much gray area here. Where is the burden of proof? Can someone argue that the depiction is of a fictional person who just looks similar to Plaintiff? How similar is too similar? Who's the judge of similarity? What if the training data did not contain any pictures of Plaintiff and the results were just tuned until it looked like him/her? What if Plaintiff has a twin sibling or someone who looks just like him/her and that person consented? What a mess--who's going to litigate all of this?
I don’t feel like anything is implying similarity is sufficient to make an artwork criminal.
Although I might call out the opposite problem of making it a crime to request a sfw picture of a specific person and have the model generate something porny
There's so much gray area because some people desperately wants porn banned than abuses and harms punished, and clarifying gray areas all the way removes side effects on purely generative porn.
Personally I think head-swap porns should be handled with defamation logic than general anti-porn laws. I find it more likely that I would be disgusted by deepfake porn of mine if I were to become such public figure that warrants it, than appreciating it, but it feels weirder that local police would be deciding whether am I to like it or not.
“It’s essential that the police and prosecutors are equipped with the training and tools required to rigorously enforce these laws in order to stop perpetrators from acting with impunity.”
We don't even have a tool that could reliably ID the subject of such a processed visual depiction.
The justice system is a computer made out of humans executing written programs called laws. As we've seen from decades of computer vision research, writing out an image classifier in a finite set of symbols is intractable. So instead the justice system queries an oracle (a human) to do the image classification.
Maybe in the future there will be a way to compile a learned image classifier into a set of directions in English and we will have laws that are followed by pages of what is essentially a small artificial neural network represented as an English flowchart, but I doubt it, that sounds dumber than asking a person to do it. You just have to make sure you're picking the right person to do the deciding.
Where do you draw the line? What about if someone, without consent, tells a story and describes every last detail of that person, short of drawing a physical picture? What if they write down that story? What if that story is kept in a safe inside your house and nobody else reads it? Why draw the line at a physical image?
It’s not really, though, is it? It’s similar to things like death threats: creating a picture of someone getting killed is not illegal, but using it in a coercitive way very much is. It’s not a completely different thing just because it involves boobs. Harm is not done when the file is created, it’s done when it is spread maliciously to bully someone, and that’s where prosecution should be focused.
The fact that a representation of a thing itself is illegal is quite backwards and ought to be at least seriously debated more seriously than in the moral panic we have now. Otherwise, we’re back to punishing people for their creations, whether that would be ideas, writings, paintings, or digital files, just because some people do not like it.
So if you have Neuralink (or anything similar that gets released in the future) you can be convicted of thoughtcrimes and that's a-ok?
What exactly is "on you" in this scenario? In your world is someone with Neuralink allowed to take their brain images and use that as an outline for a sketch they're drawing or you're not allowed to think of anyone naked ever again?
I have fundamentally bigger issue with distribution than creation in this case... Create what ever you want on your private devices... But once you share or distribute without attached consent letter you should be punished.
At least in the USA, our Supreme Court has basically legalized bribery. It's a 9-0 consensus, in their mind bribery is just how democracy works. This is what happens when ethics rules don't apply or are repeatedly ruled unconstitutional (something something fox guarding the henhouse), your term is for life, and you wield silly amounts of unchecked power.
I think you're right that there is a ton of focus on sexual imagery but not enough thought being put into the wider context of how easy it is to fake imagery (and soon audio and video). Plenty of people would still be disturbed to see fake videos of themselves fully clothed.
How are people thinking about this kind of attack? It was already viable to Photoshop someone's face into existing pornography. New tooling just lowers the barrier to execution. It seems like fighting an impossible battle since the tools are already widely available.
I feel bad for people who are harmed by the creation and publication of such content, but it's not obvious to me that it's possible to combat it effectively using the legal system. Then again, maybe such laws don't have to be completely effective, so maybe it's sufficient that sharing such content cannot be done publicly in order to deter the most obvious abuses.
Do you think in the very long term we'll remain so attached to protecting our likeness or will continued digitization of reality bring forth a loosening of such cultural norms?
> It seems like fighting an impossible battle since the tools are already widely available.
People can do lots of things easily and are prevented by the legal system. I don't agree with the part of the law that makes this a crime if it isn't shared (who is being hurt in that instance?) but using an argument that essentially says 'it is hard to prevent people from doing something so don't even bother criminalizing it' is nonsense.
Photoshop takes time and skill. AI takes a single source image and like 20 seconds. In the time it takes someone to come up with something in Photoshop the AI guy has made thousands of images.
The barrier to creation is now far, far lower. Just a few face pictures and some idea of their body type, and you've got an instant nude.
Children younger than 10 need to be taught the concept of consent and privacy, and how upsetting it is for people whose privacy and consent are violated. It ties into cliques and bullying, too. Teens can be brutal in their bullying and fake nude pictures will be added to the arsenal of abusive actions.
Some of this stems from unrestricted usage of computers by minors who don't understand the impact their actions might have. Better parental tools and parents/guardians who take this topic seriously would be beneficial too. (I'm not arguing for some locked-down console instead of standard PCs, I just think that parents need to be able to know what their kids are doing online)
That's not a good argument for creating this legislature. It couldn't really be enforced before (photoshopped images of celebrities), and it certainly can't be enforced now when any "script kiddie" will be able to generate terabytes worth of deepfakes in a week, and launch them to circulate on the internet anonymously.
Unless you institute some draconian measures like only being able to access the internet with a passport and having 10% of your workforce work in censorship.
> New tooling just lowers the barrier to execution
The word "just" here is doing a lot of work considering scale is usually the difference between something being harmless and dangerous. I think this is a blind spot with technical types as to the intent of the law. We ban things all the time that are easy to do, like shoplifting or graffiti. It's not a failing of those laws that marker pens are cheap and blank walls are everywhere.
Worth noting, I haven't read the proposed law in question and don't have an opinion on it yet. I just don't think the reasoning of something being easy means we shouldn't attempt to legislate it if it has demonstrable harm to society.
> It was already viable to Photoshop someone's face into existing pornography. New tooling just lowers the barrier to execution.
Consider libel laws. (Because, if you think about it, "explicit deepfakes" are essentially "visual libel" — scandalous claims made through visual rather than written "evidence.")
For a statement to be considered libellous, it has to be false, known as false to the writer/distributor of the statement, and written/distributed with an intent to cause harm.
But there's an implicit proposition there — that certain statements can cause harm. Which has its own requirements: for a statement to be potentially harmful, it has to make a claim that if believed would result in reputational damage; and it has to be convincing / be likely to be believed by the common man.
A statement isn't libellous if it's prima-facie false — that is, if nobody is likely to believe it.
Photoshopping someone's face onto some random person's body is easy enough to spot, that it has usually been considered (the visual equivalent of) a prima-facie false statement.
A deepfake is only bad insofar as it becomes increasingly impossible to spot the problems with it, and so becomes (the visual equivalent of) a prima-facie true statement — actual libel.
> I feel bad for people who are harmed by the creation and publication of such content, but it's not obvious to me that it's possible to combat it effectively using the legal system.
This law is about giving victims the means to combat the harm done to them
Regulating adult entertainment doesn’t work well, never has. They sites will just move their
Operating jurisdiction and move on. The only entity possibly capable would be the card issuers/processors.
This is true the world over. For example women in many states of America are fighting for their right for abortion.
When the world is prosperous, people tend to vote for more liberal policies. And when economy is struggling, people tend to vote more conservatively. So we see more authoritarian policies like these.
> Is this instance liberal (accepting of change) or conservative (hesitant of change)?
I don’t think this framing is very useful. Modern conservatives don’t have problems with change, they just want that change to go their way. They are perfectly able to innovate and push new doctrines or legal theories when it suits them. And on the other hand they do not care about preserving anything besides an idealised vision of “traditional values”, which are not as old as they seem.
Looking at the actual policies and arguments, the spectrum is between oligarchy (or dictatorship for the most extremes) and democracy. It makes much more sense that way.
> On one hand it seems liberal, wanting to see a change that gives individuals who might be subject of deepfakes more autonomy.
It’s also a significant limitation of personal freedom in the case of a victimless crime, which is anything but liberal.
Who you have described are reactionaries, not conservatives.
In fairness, some political parties who subscribe to reactionaryism operate under a capital C Conservative banner to try and muddy the waters, but to consider them conservative because of that is like considering the Democratic People's Republic of Korea democratic.
> Who you have described are reactionaries, not conservatives.
Indeed, that’s exactly what they are. But they prefer calling themselves conservatives. It’s better from a marketing perspective.
To be fair, there are not many politicians that could be called old-fashioned conservatives in the US. Most of them are a subset of the republican establishment and they were mostly wiped out in the last couple of years.
> to consider them conservative because of that is like considering the Democratic People's Republic of Korea democratic.
Definitely! Unfortunately, that’s how the semantics went, though. In the same way as what most people call “liberals” now are very different from who liberals used to be (they used to be all about capitalism and free enterprise, for example).
Magna carta only created rights for the oligarchs of the time, many of which have descents today with the same inherited wealth. You don't have to dig that deeply in the UK society to find the old aristocracy alive and well and still above others and usually the law.
There were things like the old Roman laws that defined the rights, privileges, and obligations of Roman citizens. Or the various constitutions of the old Greek poleis, which similarly guaranteed the freedom of the citizens. Or even the Hammurabi code, if you squint a bit.
“A decent job” is subject to interpretation, but I find very difficult to argue that the Magna Carta was the first, even if it was very significant.
Agreed, the capacity for real world harm these things have is enormous, while simultabeously the benefit is near non existent. The entitlement of some people to think they should have the right to make photorealistic sexual material of their unconsenting obsession is bewildering.
But your argument was wrong. People do not have the 'right' to do as they please with the image of another person. Plenty of constraints to speech exist, some are silly, some very reasonable.
Fortunately most lawmakers, for all their problems and issues, don't give a damn about pendantic nerds and will steamroll right over that argument - and rightfully so.
This is one of those "I never agreed until it happened to me" type things.
While there'll be people arguing that you can always draw someone, or cut out their face from a photo and put it on someone else the major wrong that this is attempting to right is the abundance of services that now easily automate this.
For those who talk about privacy in your own home, there's plenty of stuff that's done in peoples homes that's illegal and no one cares about. This law's only going to come into effect when these start being shared which I hope we can all agree is a bad thing if done unconsensually.
I'm intrigued by similar state-level laws being passed in the US; a lot of those seem unconstitutional (1A/free speech) in the US, unless incredibly narrowly tailored and banning publishing or other harms from use.
More reinforcement that the UK lacks a lot of protections of basic human rights, though.
The Criminal Justice and Courts Act 2015 makes distributing intimate images without consent a crime in England and Wales. It prohibits sharing, or threatening to share, private sexual images of someone else without their consent and with the intent to cause distress or embarrassment to that person.
I get that deepfakes are superimposing a non-consenting face on a (presumably) consenting body, but how does that matter? Is it that "visually lying" is a legal loophole?
> I get that deepfakes are superimposing a non-consenting face on a (presumably) consenting body, but how does that matter? Is it that "visually lying" is a legal loophole?
That's not how deepfakes work, since nothing is superimposed.
Models like Stable Diffusion build a training set out of several images of a face (e.g. with something like LORA), and reduce it to a set of algorithms. That algorithm is a sophisticated way of expressing "green shirt, black hair, large chin, etc" It's not actually storing photos of faces. It then unmasks a new face out noise using those blueprints. You can change the blueprints, for example, I could ask SD to give me an image with gigantic rabbit ears.
To a layperson it might look the same since the technology is so new, but it's not in any way analogous to a photograph, it's analogous to a painter who saw someone, wrote down some notes on a notepad, went to their studio and painted them. The painting might look realistic but it's not the same as the original photo.
> The minister for victims and safeguarding, Laura Farris, said the bill sent the message that making deepfake images was ‘immoral and often misogynistic’
What is misogynistic is thinking that women's bodies are so shameful that using a computer algorithm to generate visual depictions of them without any interaction with the woman is inherently degrading or inherently harmful.
Yes, they can be used for harassment. But so can a text message "XXX is a sl*t". So can a crude pencil sketch.
The misogyny is in the intention and use, not any sense of realistically depicting the female body.
I wonder if look-a-like people can license their images for this or if you can license a deepfake and make it legal? I have a feeling things are going to get murky here. Hopefully we can all be mature and caring people but I have my doubts.
First let me say that I fully understand how the leaking of actual intimate photos/videos can be very devastating. No arguments there.
But AI porn? There are so many gray areas and loopholes. This just feels so friggin' half-baked. It just feels like something that should be handled on a case by case basis. If somebody creates AI porn of somebody and uses it for clearly harmful purposes (blackmail, humilating somebody to their coworkers, whatever) then that should be punished.
But, outlawing them altogether? We tolerate pornographic fanfiction and traditional pornographic illustrations of real adults, even though these might be upsetting to the people depicted therein.
Therefore the issue with deepfakes is that not only are these things upsetting, but they might be mistaken for actual photos and videos.
- What if I create deepfake porn of $FAMOUS_ACTRESS but it's sort of crappy, and nobody could reasonably mistake it for an actual photo of $FAMOUS_ACTRESS?
- What if I create deepfake porn of $FAMOUS_ACTRESS and it's extremely photorealistic, but has clearly fantastical elements? (e.g. she is naked on the surface of the moon, she is having sex with Godzilla, etc)
- What if I create deepfake porn of $FAMOUS_ACTRESS and it's realistic, but I've clearly indicated via caption or some other means that it's a synthetic image and not an actual photo of $FAMOUS_ACTRESS?
- What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate out her face?
- What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate the explicit parts?
- How explicit is explicit? What if I've depicted $FAMOUS_ACTRESS in a swimsuit or other revealing outfit that is perfectly legal to wear in public? (reminder: going topless or even nude is legal in a lot of places too)
- What if I've generated AI porn of $FAMOUS_ACTRESS that is based on publicly available images? eg: nude scenes from her movies, etc
- What if I create deepfake porn and you can't tell who it is? Do we go on a search for the victim?
- What if I create AI-generated porn and it accidentally looks like somebody real?
- What if I create deepfake porn of $FAMOUS_ACTRESS but I never say it's her?
- What if I create deepfake porn of $FAMOUS_ACTRESS but I change a detail or two? Perhaps I depict her with the "wrong" eye color, or add a scar, or something. So is this deepfake porn of "her" or more of a derivative work?
"The creation of a deepfake image will be an offence regardless of whether the creator intended to share it"
Surely the basis of a lucrative new extortion model. The most dangerous computer virus will be one that creates an image on your PC and alerts the extortionist.
The criminal offence refers to deepfakes of a real person without their consent. Actual people can be identified in this AI generated content, where the resemblance is close / beyond doubt.
----
The harm:
Many people do not seem to understand the harm. Deepfakes are used for sexual pleasure, and used to degrade and dehumanize people resulting in victims getting traumatised. Victims often do not get enough support and the trauma may stay for a lifetime, and some victims may end their lives due to this.
It is worth to discuss the 'thoughtcrime' where the generated content is not shared and where the deepfaked person is not harassed. Quote from the article:
> The creation of a deepfake image will be an offence regardless of whether the creator intended to share it [..]
• Revenge Porn does not make it illegal to store pornographic content as long as there was consent. However, Revenge Porn prohibits sharing no longer consensual content.
• Sharing the generated content is not necessary in order to degrade, dehumanize and harass victims, thus still cause harm. On the other hand, if the deepfaked people are not harmed then it is a bit different, which may seem like a "thoughtcrime" and possibly may be.
• People cannot guarantee the data they store won't get leaked. Both Revenge Porn and Deepfakes have the same issue. However, in Revenge Porn it is easier to see who created the content and enforce the law, while in deepfakes harder.
131 comments
[ 0.21 ms ] story [ 80.8 ms ] thread[1] https://www.gov.uk/government/news/government-cracks-down-on...
This is an interesting one because it violates the vague principle of "whatever done by consenting adults in the privacy of their own home is cool".
Like what if someone pastes my face ona naked body, or imagines me naked, or has a dream about me flying through the window naked and writes it in their private dream diary.
I think it's getting into hairy legal territory if the "infringing material" is only seen by the otherwise innocent creator on an airgapped computer.
Instead of looking for those answers in the text of Farris’ summary of the law, we should look at the text of the law itself.
Edit: https://bills.parliament.uk/bills/3511/publications probably has the text we're looking for, somewhere.
A private deepfake image is not the same.
There are plenty of incredibly disturbing videos and images of some of the most gruesome crimes committed and they are lawfully accessible and obtainable on the open Internet, including some such images on Wikipedia. The documenting of a crime is not and has never been illegal.
On a related note, I assume there is already a lot of deepfake CSAM in certain circles. If that is ruled legal, it will make finding and prosecuting real life CSAM creators much more difficult.
At least in the US.
Well if those images are illegal, you're right that there's no way to produce them legally, but that's because they are on their face illegal.
If you mean there is no way to produce CSAM that doesn't break other laws, I think you're wrong - e.g. see the case of teenagers taking nude pictures of themselves. That is an otherwise legal thing to do, but having those images is illegal, as I understand it, and you can search Google for many examples of teens prosecuted for sexting images of themselves. I don't know if anyone has been prosecuted for merely having nude images of themselves, but that might just be because no one would even know about it, necessarily.
https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...
I wonder if deepfake technology is going to change our stance on that. With enough motivation Congress could take it out of the hands of SCOTUS.
https://en.m.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coal...
Maybe it could be read that way by some, but that's not why the term was adopted and it's not why the term is advocated for by anti-sexual violence organizations.
https://inhope.org/EN/articles/child-sexual-abuse-material
https://www.rainn.org/news/what-child-sexual-abuse-material-...
https://www.thorn.org/blog/decoding-the-language-of-child-se...
As for the fact that making it simulated renders the term meaningless, the entire point of a simulation is that it is not the actual thing that is happening but rather an imitative representation of something that could be real. Presumably the abuse towards a child could be real, and hence it's possible to imitate that abuse in various forms of media. It is that imitation that constitutes the simulation.
If you argue that it makes the term meaningless, then you must accept that the simulation of anything is meaningless.
I think that what is being argued. Put 'simulated' in front of any legal term and it becomes meaningless for its legal usage. 'Simulated larceny' is meaningless. 'Simulated murder' is meaningless.
There are plenty of laws that make it an offense to perform a simulation. Nitpicking the term simulation as being meaningless, whether in a legal context or otherwise, isn't a particularly noteworthy discussion to have. I think most people understand the meaning of simulated CSAM and can have an opinion on whether it should be legal or illegal without being pedantic about the word "simulated".
You are really stretching here, and it is not 'pedantic' to ask that specific terms have specific meanings especially when it is a legal issue.
You are laughably misinformed on this matter in a way that is quite incomprehensible. You are welcome to refer to the legal definition as per the actual legal text instead of continuing to make rather bizarre claims on a topic where it's clear you know nothing about:
https://www.congress.gov/bill/108th-congress/senate-bill/151...
Even without having to refer to the actual text, I am confident most reasonable minded people understand the meaning of the term "simulated" and can take a position on whether it should be legal or not without arguing whether simulation means training materials or whether the term is legally meaningless.
The fact that you don't is all I need to know that continuing this conversation with you is entirely pointless.
I think the issue here is the between consenting adults part, deepfakes in this instance are probably not between consenting adults. I also think it's important to think about the reality of how this likely plays out. If someone deepfakes me and faps and I never find out and they never tell anyone, it would be quite hard for there to be action. So if someone is deepfaking you and you know about it, there must be a logical path there, and none of them look good for the deepfaker. Can't say I know where I land on the issue, and I've thought about it for days and hours.
People are acting like a "deepfake of a person" is something different than a drawing, painting, song, poem or any other artistic creation inspired by, based on, or with elements drawn from a real person.
"If someone is deepfaking you" is such a weird imaginary problem. Someone might be imagining you. Someone might see you in a coffee shop while writing a screenplay and use the outfit you're wearing to describe a character in their story. Who cares?
If someone is committing fraud by trying to pass off a deepfake as a real photo, then we already have laws around fraud, defamation, harassment to cover that. If someone makes a deepfake that looks similar to a real person for their own amusement, that's no different than writing fan-fiction or sketching in a notebook.
Distribution of an image is not an effect of its creation.
Deepfakes in this instance are not /between/ anyone. Only one person even knows of it.
I feel like "without consent" to be pretty key here.
Which is basically what we're seeing here.
IF the artwork is porn.
In fact, if you're not old enough, it's just flat out illegal. You'll go to prison for making that artwork. And you arguing that it's "art" is not going to find much resonance in US courts at least. You're likely to fail the Miller test if US citizens are the judges.
For example, cooking meth.
I do think the other examples you gave have one significant and fundamental difference - no one would mistake those things as being real. The problem with deepfakes is these have a similar social impact and capcity for harm as real nudes do, only there is no element of consent. You could ruin someones life quite easily with deep fakes, but not with your lewd poetry or sketches. So it seems reasonable to me to ban these with a blanket ban.
I would prefer we lived in a world where that wasnt neccesary, but for every technology you must imagine the most sinister and exploitative application and realise that someone will try that and worse. How do you even begin to combat deepfakes and the social problems they bring? how many bodies are considered acceptable until we get it right? Possibly, its best to over reach a little then relax the grip rather than to take it slow, given what is on the line for some.
A ban won't stop bad people from making them anyway. On the other hand, proliferation of deepfakes would make the existence of certain real or fake photos insignificant, as everyone would accept that anyone could make them easily.
sigh.
I file this whole issue under: "This is why we can't have nice things".
People who create models like that ruin everything for everyone.
Courts have previously deemed cartoon imagery of children to be equivalent to a real child, so considering the deepfake to be a real person that needs to give consent is not without precedent.
The implied answer here appears to be “no” and the reason this is deemed necessary is an emergent effect of scale of being able to do this at such volume and quality.
Saying you can’t draw porn of people without regard to implied authenticity or medium seems somehow more reasonable to me.
Although I might call out the opposite problem of making it a crime to request a sfw picture of a specific person and have the model generate something porny
Personally I think head-swap porns should be handled with defamation logic than general anti-porn laws. I find it more likely that I would be disgusted by deepfake porn of mine if I were to become such public figure that warrants it, than appreciating it, but it feels weirder that local police would be deciding whether am I to like it or not.
We don't even have a tool that could reliably ID the subject of such a processed visual depiction.
The judge.
https://en.wikipedia.org/wiki/I_know_it_when_I_see_it
The justice system is a computer made out of humans executing written programs called laws. As we've seen from decades of computer vision research, writing out an image classifier in a finite set of symbols is intractable. So instead the justice system queries an oracle (a human) to do the image classification.
Maybe in the future there will be a way to compile a learned image classifier into a set of directions in English and we will have laws that are followed by pages of what is essentially a small artificial neural network represented as an English flowchart, but I doubt it, that sounds dumber than asking a person to do it. You just have to make sure you're picking the right person to do the deciding.
Also UK: Criminalizes 'thoughtcrime'.
The fact that a representation of a thing itself is illegal is quite backwards and ought to be at least seriously debated more seriously than in the moral panic we have now. Otherwise, we’re back to punishing people for their creations, whether that would be ideas, writings, paintings, or digital files, just because some people do not like it.
But creating an image that can be watched in the real world, is more than "thought".
What exactly is "on you" in this scenario? In your world is someone with Neuralink allowed to take their brain images and use that as an outline for a sketch they're drawing or you're not allowed to think of anyone naked ever again?
Also UK: make above real enough that it has a term
that's some admirable British consistency. Unironically. Honestly.
I feel bad for people who are harmed by the creation and publication of such content, but it's not obvious to me that it's possible to combat it effectively using the legal system. Then again, maybe such laws don't have to be completely effective, so maybe it's sufficient that sharing such content cannot be done publicly in order to deter the most obvious abuses.
Do you think in the very long term we'll remain so attached to protecting our likeness or will continued digitization of reality bring forth a loosening of such cultural norms?
People can do lots of things easily and are prevented by the legal system. I don't agree with the part of the law that makes this a crime if it isn't shared (who is being hurt in that instance?) but using an argument that essentially says 'it is hard to prevent people from doing something so don't even bother criminalizing it' is nonsense.
Children younger than 10 need to be taught the concept of consent and privacy, and how upsetting it is for people whose privacy and consent are violated. It ties into cliques and bullying, too. Teens can be brutal in their bullying and fake nude pictures will be added to the arsenal of abusive actions.
Some of this stems from unrestricted usage of computers by minors who don't understand the impact their actions might have. Better parental tools and parents/guardians who take this topic seriously would be beneficial too. (I'm not arguing for some locked-down console instead of standard PCs, I just think that parents need to be able to know what their kids are doing online)
That's not a good argument for creating this legislature. It couldn't really be enforced before (photoshopped images of celebrities), and it certainly can't be enforced now when any "script kiddie" will be able to generate terabytes worth of deepfakes in a week, and launch them to circulate on the internet anonymously.
Unless you institute some draconian measures like only being able to access the internet with a passport and having 10% of your workforce work in censorship.
No privacy is violated by this act.
Consent violation is nonsensical.
The word "just" here is doing a lot of work considering scale is usually the difference between something being harmless and dangerous. I think this is a blind spot with technical types as to the intent of the law. We ban things all the time that are easy to do, like shoplifting or graffiti. It's not a failing of those laws that marker pens are cheap and blank walls are everywhere.
Worth noting, I haven't read the proposed law in question and don't have an opinion on it yet. I just don't think the reasoning of something being easy means we shouldn't attempt to legislate it if it has demonstrable harm to society.
Consider libel laws. (Because, if you think about it, "explicit deepfakes" are essentially "visual libel" — scandalous claims made through visual rather than written "evidence.")
For a statement to be considered libellous, it has to be false, known as false to the writer/distributor of the statement, and written/distributed with an intent to cause harm.
But there's an implicit proposition there — that certain statements can cause harm. Which has its own requirements: for a statement to be potentially harmful, it has to make a claim that if believed would result in reputational damage; and it has to be convincing / be likely to be believed by the common man.
A statement isn't libellous if it's prima-facie false — that is, if nobody is likely to believe it.
Photoshopping someone's face onto some random person's body is easy enough to spot, that it has usually been considered (the visual equivalent of) a prima-facie false statement.
A deepfake is only bad insofar as it becomes increasingly impossible to spot the problems with it, and so becomes (the visual equivalent of) a prima-facie true statement — actual libel.
Which country's libel laws. They are not all the same. The UK is famous for libel actions that would not be countenanced by US courts.
This law is about giving victims the means to combat the harm done to them
Creation alone harms exactly nobody.
Source: worked in that industry for almost 20 years.
When the world is prosperous, people tend to vote for more liberal policies. And when economy is struggling, people tend to vote more conservatively. So we see more authoritarian policies like these.
On one hand it seems liberal, wanting to see a change that gives individuals who might be subject of deepfakes more autonomy.
On the other hand, AI is changing the world and it seems like an effort to try and hold the things around AI in place as to not seem them change.
It could go either way. So could the state of the economy right now, for that matter.
I don’t think this framing is very useful. Modern conservatives don’t have problems with change, they just want that change to go their way. They are perfectly able to innovate and push new doctrines or legal theories when it suits them. And on the other hand they do not care about preserving anything besides an idealised vision of “traditional values”, which are not as old as they seem.
Looking at the actual policies and arguments, the spectrum is between oligarchy (or dictatorship for the most extremes) and democracy. It makes much more sense that way.
> On one hand it seems liberal, wanting to see a change that gives individuals who might be subject of deepfakes more autonomy.
It’s also a significant limitation of personal freedom in the case of a victimless crime, which is anything but liberal.
In fairness, some political parties who subscribe to reactionaryism operate under a capital C Conservative banner to try and muddy the waters, but to consider them conservative because of that is like considering the Democratic People's Republic of Korea democratic.
Indeed, that’s exactly what they are. But they prefer calling themselves conservatives. It’s better from a marketing perspective.
To be fair, there are not many politicians that could be called old-fashioned conservatives in the US. Most of them are a subset of the republican establishment and they were mostly wiped out in the last couple of years.
> to consider them conservative because of that is like considering the Democratic People's Republic of Korea democratic.
Definitely! Unfortunately, that’s how the semantics went, though. In the same way as what most people call “liberals” now are very different from who liberals used to be (they used to be all about capitalism and free enterprise, for example).
“A decent job” is subject to interpretation, but I find very difficult to argue that the Magna Carta was the first, even if it was very significant.
Also, what you say is very true. The Magna Carta was a hastily cobbled document to protect the power balances and keep the serfs in their place.
That's japan.
When laws are changed with careful consideration, for both individual rights and social good – well, that's the very essence of the social contract.
It is a valid argument where I used it - against your purported bewilderment.
That's not the right under discussion.
The right is, in your own words: "the right to make photorealistic sexual material of their unconsenting obsession".
Your exaggeration of the scope does your argument no favour.
This is one of those "I never agreed until it happened to me" type things.
For those who talk about privacy in your own home, there's plenty of stuff that's done in peoples homes that's illegal and no one cares about. This law's only going to come into effect when these start being shared which I hope we can all agree is a bad thing if done unconsensually.
Not at all. Else the criminalisation would be of those services.
More reinforcement that the UK lacks a lot of protections of basic human rights, though.
And there are plenty of people hell bent on removing those that remain.
I get that deepfakes are superimposing a non-consenting face on a (presumably) consenting body, but how does that matter? Is it that "visually lying" is a legal loophole?
That's not how deepfakes work, since nothing is superimposed.
Models like Stable Diffusion build a training set out of several images of a face (e.g. with something like LORA), and reduce it to a set of algorithms. That algorithm is a sophisticated way of expressing "green shirt, black hair, large chin, etc" It's not actually storing photos of faces. It then unmasks a new face out noise using those blueprints. You can change the blueprints, for example, I could ask SD to give me an image with gigantic rabbit ears.
To a layperson it might look the same since the technology is so new, but it's not in any way analogous to a photograph, it's analogous to a painter who saw someone, wrote down some notes on a notepad, went to their studio and painted them. The painting might look realistic but it's not the same as the original photo.
What is misogynistic is thinking that women's bodies are so shameful that using a computer algorithm to generate visual depictions of them without any interaction with the woman is inherently degrading or inherently harmful.
Yes, they can be used for harassment. But so can a text message "XXX is a sl*t". So can a crude pencil sketch.
The misogyny is in the intention and use, not any sense of realistically depicting the female body.
But AI porn? There are so many gray areas and loopholes. This just feels so friggin' half-baked. It just feels like something that should be handled on a case by case basis. If somebody creates AI porn of somebody and uses it for clearly harmful purposes (blackmail, humilating somebody to their coworkers, whatever) then that should be punished.
But, outlawing them altogether? We tolerate pornographic fanfiction and traditional pornographic illustrations of real adults, even though these might be upsetting to the people depicted therein.
Therefore the issue with deepfakes is that not only are these things upsetting, but they might be mistaken for actual photos and videos.
- What if I create deepfake porn of $FAMOUS_ACTRESS but it's sort of crappy, and nobody could reasonably mistake it for an actual photo of $FAMOUS_ACTRESS?
- What if I create deepfake porn of $FAMOUS_ACTRESS and it's extremely photorealistic, but has clearly fantastical elements? (e.g. she is naked on the surface of the moon, she is having sex with Godzilla, etc)
- What if I create deepfake porn of $FAMOUS_ACTRESS and it's realistic, but I've clearly indicated via caption or some other means that it's a synthetic image and not an actual photo of $FAMOUS_ACTRESS?
- What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate out her face?
- What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate the explicit parts?
- How explicit is explicit? What if I've depicted $FAMOUS_ACTRESS in a swimsuit or other revealing outfit that is perfectly legal to wear in public? (reminder: going topless or even nude is legal in a lot of places too)
- What if I've generated AI porn of $FAMOUS_ACTRESS that is based on publicly available images? eg: nude scenes from her movies, etc
- What if I create deepfake porn and you can't tell who it is? Do we go on a search for the victim?
- What if I create AI-generated porn and it accidentally looks like somebody real?
- What if I create deepfake porn of $FAMOUS_ACTRESS but I never say it's her?
- What if I create deepfake porn of $FAMOUS_ACTRESS but I change a detail or two? Perhaps I depict her with the "wrong" eye color, or add a scar, or something. So is this deepfake porn of "her" or more of a derivative work?
Surely the basis of a lucrative new extortion model. The most dangerous computer virus will be one that creates an image on your PC and alerts the extortionist.
---- The harm:
Many people do not seem to understand the harm. Deepfakes are used for sexual pleasure, and used to degrade and dehumanize people resulting in victims getting traumatised. Victims often do not get enough support and the trauma may stay for a lifetime, and some victims may end their lives due to this.
These victims get sexually harassed, e.g. one woman is continuously being harassed after someone generated deepfakes of her and people recognised her from those images https://www.reddit.com/r/legaladvice/comments/1c1pzjo/ai_gen...
In r/singularity thread this is discussed and unfortunately many people think it's okay. https://www.reddit.com/r/singularity/comments/1c5c8pp/uk_cri...
---- Thoughtcrime:
It is worth to discuss the 'thoughtcrime' where the generated content is not shared and where the deepfaked person is not harassed. Quote from the article:
> The creation of a deepfake image will be an offence regardless of whether the creator intended to share it [..]
• Revenge Porn does not make it illegal to store pornographic content as long as there was consent. However, Revenge Porn prohibits sharing no longer consensual content.
• Sharing the generated content is not necessary in order to degrade, dehumanize and harass victims, thus still cause harm. On the other hand, if the deepfaked people are not harmed then it is a bit different, which may seem like a "thoughtcrime" and possibly may be.
• People cannot guarantee the data they store won't get leaked. Both Revenge Porn and Deepfakes have the same issue. However, in Revenge Porn it is easier to see who created the content and enforce the law, while in deepfakes harder.