It's because the deep fake is literally trying to be a forgery of a real pornographic act where a hand drawing is obviously not a forgery or a real pornographic act
Have fun trying to get a legal definition for that. The evil machine just copies what it has seen before to create a composite, while the human 3D-artist just draws from what they know. Not to mention that the rules around what's considered 'art' is extremely murky; e.g. the band Scorpions feature a nude minor on one of their album-covers which is fine "because art" while people have gone to jail for having hentai featuring underage fictional characters.
Is there a clear bright-line between a cartoon or South Park-style animation, a hand-drawn photorealistic graphic, and a ML-generated one that should be legislated differently than existing laws against commercial usage of somebody's likeness vs fair use?
If so, what is that bright-line, and what unforeseen consequences might legislating it have?
I recognize the limits on regulating free speech are weaker in the UK where this story originated as opposed to in the US.
Given that colored pencil drawings can be photorealistic, I'm not sure a "bright line" exists. Deepfakes just make it easier to do something that has likely been done since the creation of drawing tools.
I would guess something akin to existing copyright/branding rules?
Something like "can an unknowing person genuinely confuse the copy with the original, and did the person who produce the copy did it to exploit such confusion".
Is that the existing copyright rule? It might be the existing trademark rule, but i don't think that is the existing copyright rule.
Arguably from a copyright perspective deepfakes might be derrivitive works of the used source material, but that is generally owned by the photographer not the model.
Probably personality rights would be the way to go here, rather than trademark or copyright.
There doesn't have to be a clear bright line, even when the law seems clear the reality is many times fuzzy. This is why we have judges, to make a ruling in the spirit of the law.
In this case, if the likeliness is close enough then it's forbidden.
Not one that represents a universal truth - there never is. We decide on lines that are arbitrary to reality but interesting to humans because we have no other choice. Note that this is different from subjective vs objective, which is a second axis, but is often used as a synonym for the first when creating a false dilemma. Objectivity is more relevant when deciding on the written language of the implementation, which is different from the interesting moral problem.
Likeness rights are a thing, and that thing works a lot like trademarks/“trade dress.”
If a picture could be confused for being something created using a given model as a reference input, then the model has a right to claim a violation of their likeness rights. Just like a can of soda that tries to confuse you into thinking it’s Coca-Cola, without ever saying “Coca-Cola” anywhere on it, is violating Coca-Cola Corporation’s trade-dress rights.
in general, you are risking a lawsuit if you use anyone's likeness without their permission, especially if you depict them in a negative light and/or for commercial purposes. parodies are protected (if they are sufficiently obvious like south park), and it is usually hard for a public figure to win such a suit unless they can show that you invaded their privacy.
free speech enjoys strong protections in the US, but so does copyright. I'm not sure why the article acts like all the legal processes available are onerous and/or inapplicable. it's not complicated to file a DMCA takedown, and it would almost certainly result in these videos being taken down with minimal review.
of course, you would likely see the same thing as with other copyrighted material. you can push it off the main sites where it gets the most views, but if people really want to upload and view something, they will find a place to host it. you can either accept this, or you can push for more draconian laws that don't fix the problem, but make everyone's lives a little more difficult.
The ability to create a realistic pornographic photo using someone else's likeness has been around for a long time; why does the medium of video make things different? We've got to let go of our notion that video is reality and can't be faked. The time when that was true is definitely gone.
There's a difference between being possible and trivial. What once required professional artists can now be done by kids with lots of free time. That changes the dynamic and new problems arise.
There are plenty of things that society finds tolerable when they happen in small amounts, but problematic when they happen in larger amounts.
That society ignored faked Britney Spears nudes in 1997 but complains about deep fakes today is no more inconsistent than someone who likes St Patrick's Day parades complaining when his neighbour turns his home into a 24-hour bagpipe practice centre.
I'd like to consider your argument but I'd appreciate if you could come up with a better example than the one you've given. Why can society tolerate a pornographic painting or drawing of a celebrity, a photoshopped nude or pornographic photo, but not a faked pornographic video?
So it seems to me that there are two types of people: those who don't care if they're seen naked and those who do. Among those who don't care, this wouldn't be a problem, and for those that do, this actually works in their favor if it becomes widespread: if their personal pictures are leaked, they have perfect plausible deniability.
Many who are willing to appear naked expect to be paid for it, just as an athlete who lets his image be used in an ad isn't giving everyone else permission to photoshop his image into their ads for free.
This argument doesn't work in more traditional societies. There was a actress in India who had demonstrations against her because someone morphed her face onto a nude pic.
The kinds of people who work for Pornhub aren’t going to have moral qualms about this[1] and there’s no way you can ban deep fakes as such under the First Amendment. Either there will be regulation of porn on grounds of obscenity or the current free for all will continue but there’s no way deep fakes will be banned and the rest of porn left alone.
It seems likeness rights might be sufficient? You can't put random college athlete's in a video game, the situations aren't exactly the same but I don't see many 1st amendment purists even talking about likeness rights.
>>You can't put random college athlete's in a video game
I'm pretty sure you can, as in - the law doesn't forbid you from doing so. But the person you are portraying may and will sue you for damages and/or portion of your profits. But that would be a civil case, not a criminal one, right?
No? If you sign an agreement with someone, and then break it, no law has been broken. The other party can sue you for breaking the contract in civil court and win, but there is no law that says you cannot break agreements signed with someone. Of course there might be cases in civil court which are about law violations, but it's not a universal rule.
I got a popup that said "PORN KILLS LOVE" TM copyright by "Fight the New Drug, Inc" and then I thought "ooh this is propaganda" and didn't actually read the article.
Commercial speech is subject to more regulation than non commercial speech. Likeness rights in commercial speech are already a thing.
Also if you aren't very explicit in saying that this is not really a famous celebrity, then you're basically defaming that celebrity, which is also not protected speech.
An appalling story - but the legitimate news source (the BBC - the other link is a campaigning blog which omits relevant information) makes it clear that PornHub have fully acknowledged they completely failed the victim here.
Its not clear how you move from that story to "The kinds of people who work for Pornhub aren’t going to have moral qualms about this", and even less clear what this has to do with DeepFakes.
There is clearly a real problem with people sharing non-consensual real material; we probably shouldn't muddy the water by mixing it with the phenomenon of DeepFakes - the morality of which is pretty much just a matter of opinion.
I am wondering about the implications of deep image generation when it comes to child pornography. It is one of the few kinds of speech that is not protected under the First Amendment.
How would it be possible to tell an image created with image2text or similar was generated from a model trained on images of child pornography? In this case it would probably still be illegal. Although nobody is harmed from the generation of such an image the courts would probably only care about where the source of the image originated, as with copyright.
Or what about using the likeness of a child on the body of a consenting adult for a deepfake? I that case nobody is harmed to begin with.
There's probably a legal argument that a deepfake of someone doing something scandalous (eg starring in pornography) counts as libel.
> Libel is a method of defamation expressed by print, writing, pictures, signs, effigies, or any communication embodied in physical form that is injurious to a person's reputation, exposes a person to public hatred, contempt or ridicule, or injures a person in his/her business or profession. [1]
A pornographic deepfake is certainly a "defamation expressed by [...] picture [...] that is injurious to a person's reputation" insofar as it creates a false impression about the person which is detrimental to their reputation.
The article mentions a Czech holding company owning some of these sites. The amount of porn, both straight and gay, produced in Czech Republic is astounding. You can't live in Prague without being harassed by the industry. I received a cease and desist letter for physically pushing some porn producers harassing people on the busiest street in Prague.
As a regular porn watcher myself and in many cases a defender of free speech (except in the case of hate speech coming from fascists) I still have to admit that the destructive capability of deep fakes of public figures is deeply concerning. Female politicians like AOC should not have to deal with malicious actors spreading pornographic deep fakes. Some may remember the countless lewd doctored photos of Hillary Clinton that spread in conservative circles. It only stands to reason that this helped the current president win, and in general makes it harder for all female politicians to win. This doesn’t seem to affect male politicians nearly to the same degree.
Simultaneously these porn companies are making exorbitant amounts of money from obviously illegal content. Something needs to be done but at the same time I worry this could be a political opportunity for a conservative to push forward an internet censorship bill.
While this is abhorrent, the likely end result is people will stop trusting video they see online. That combined with sites like thispersondoesnotexist.com might actually increase the personal privacy of people online. We might end up in a state where employers stop researching candidates from their apparent Facebook profiles.
Really? To me, it seems not optimistic but inevitable. I don't think anyone believes Emma Watson has actually done a porn, despite her appearing in the deepfake mentioned in the article. Being deepfaked is unlikely to affect anyone's real-world prospects in any way. That being said I can understand that it would still be hurtful and I'm open to having the content more tightly regulated.
Are you saying that you think folks believe Emma Watson has done a porn, or something else? I mean, the claim I'm making is empirical. If you went out and asked people, you could know the answer.
You have made two claims directly, and asserted their equivalence for a third claim.
People have not seen the Emma Watson porn, and those that have can probably read the word deepfake in the title. My guess is between 5% and 20% of the people who have seen it, and found it through searching for Emma Watson, rather than Emma Watson Deepfake believe that they have seen actual Emma Watson in porn.
But ultimately, you said that you didn't believe that it was irresponsible to assert that deepfakes will society more skeptical of video. That is your claim. I pose that deepfakes can easily be used to produce videos which are closer to what their audiences expect than reality, and as a result they will likely be less suspicious of what they see.
Kind of. But realise when this starts happening at-scale, every news anchor, celebrity and politician will be talking about the dangers of deepfakes. The layperson won't be ignorant for long. Already I see non-techie friends talking about deepfakes.
Are these deepfakes presented as real records, or acknowledged as deepfakes? In the second case, there is not much difference compared to porn movies with actors reresenting politicians, e.g.: https://en.wikipedia.org/wiki/Who%27s_Nailin%27_Paylin%3F
The deepfakes I’ve seen are often presented without context and in many cases are assumed to be real if done well. This ability to create convincing deepfakes will only improve over time.
Even if factually deepfakes don’t substantially differ from impersonations, on a moral level they should not be tolerated, especially for grass roots female politicians.
What a weird and misleading article. I assume it was written by that "deepfake detection company Sensity" to drum up business, but it certainly doesn't seem to be well-researched.
As is, it is already illegal to use Emma Watson's "likeness" without her approval in pretty much every western country, including the EU and the Czech republic. "likeness" here means that the fake is good enough for regular people to recognize the actress.
The article continues: “Until there is a strong reason for [porn websites] to try to take them down and to filter them, I strongly believe nothing is going to happen,” Patrini says.
Obviously. That "strong reason" is called a lawsuit and is pretty easy to organize. You might even sue for punitive damages and recoup your legal costs, meaning a celebrity successfully suing might be as expensive as $350,000 per video. I'd say that is a very strong reason to take things down, if - and apparently only if - the person requesting the takedown is following legal procedures.
"Under California law, intentional imitation of a celebrity’s distinctive and widely known voice for commercial purposes constitutes tortious misappropriation."
Intentional is the key word here. So I presume that if you look like Emma Watson and go with your real name, you're fine. If you pretend to be Emma Watson, then it becomes infringing.
Depends on your intention. If you produce the video with the intent to profit off her fame, and that will certainty be assumed if you use her name in the tags or keywords, then it's clearly illegal.
But if the video says "this is a deepfake" and is not showing up in search results for her name, then it should be fine.
Its just that in the latter case, probably nobody is going to watch it, so the discussion is then moot anyway.
67 comments
[ 4.0 ms ] story [ 121 ms ] threadI confess I hadn't considered that angle :D
If so, what is that bright-line, and what unforeseen consequences might legislating it have?
I recognize the limits on regulating free speech are weaker in the UK where this story originated as opposed to in the US.
Something like "can an unknowing person genuinely confuse the copy with the original, and did the person who produce the copy did it to exploit such confusion".
Arguably from a copyright perspective deepfakes might be derrivitive works of the used source material, but that is generally owned by the photographer not the model.
Probably personality rights would be the way to go here, rather than trademark or copyright.
IANAL.
Not one that represents a universal truth - there never is. We decide on lines that are arbitrary to reality but interesting to humans because we have no other choice. Note that this is different from subjective vs objective, which is a second axis, but is often used as a synonym for the first when creating a false dilemma. Objectivity is more relevant when deciding on the written language of the implementation, which is different from the interesting moral problem.
If a picture could be confused for being something created using a given model as a reference input, then the model has a right to claim a violation of their likeness rights. Just like a can of soda that tries to confuse you into thinking it’s Coca-Cola, without ever saying “Coca-Cola” anywhere on it, is violating Coca-Cola Corporation’s trade-dress rights.
free speech enjoys strong protections in the US, but so does copyright. I'm not sure why the article acts like all the legal processes available are onerous and/or inapplicable. it's not complicated to file a DMCA takedown, and it would almost certainly result in these videos being taken down with minimal review.
of course, you would likely see the same thing as with other copyrighted material. you can push it off the main sites where it gets the most views, but if people really want to upload and view something, they will find a place to host it. you can either accept this, or you can push for more draconian laws that don't fix the problem, but make everyone's lives a little more difficult.
Photoshop made a lot of stuff accessible that previously wasn't.
In a few decades, we'll merely tell the computers what we want to see and they'll make it for us.
In fifty, they'll infer what we want and dynamically adjust.
In a hundred years, all reality will be constructed.
The trend is bigger and more important than the fear.
We can choose to embrace this change or take measures to make it less trivial.
That society ignored faked Britney Spears nudes in 1997 but complains about deep fakes today is no more inconsistent than someone who likes St Patrick's Day parades complaining when his neighbour turns his home into a 24-hour bagpipe practice centre.
Reddit would demonstrate otherwise. As would Burning Man, nude beaches, and any number of other locations where nudity is allowed/expected.
[1] https://fightthenewdrug.org/pornhub-refused-to-remove-videos...
https://www.bbc.com/news/amp/stories-51391981
I'm pretty sure you can, as in - the law doesn't forbid you from doing so. But the person you are portraying may and will sue you for damages and/or portion of your profits. But that would be a civil case, not a criminal one, right?
Edit: I was wrong :)
I got a popup that said "PORN KILLS LOVE" TM copyright by "Fight the New Drug, Inc" and then I thought "ooh this is propaganda" and didn't actually read the article.
Also if you aren't very explicit in saying that this is not really a famous celebrity, then you're basically defaming that celebrity, which is also not protected speech.
Its not clear how you move from that story to "The kinds of people who work for Pornhub aren’t going to have moral qualms about this", and even less clear what this has to do with DeepFakes.
There is clearly a real problem with people sharing non-consensual real material; we probably shouldn't muddy the water by mixing it with the phenomenon of DeepFakes - the morality of which is pretty much just a matter of opinion.
How would it be possible to tell an image created with image2text or similar was generated from a model trained on images of child pornography? In this case it would probably still be illegal. Although nobody is harmed from the generation of such an image the courts would probably only care about where the source of the image originated, as with copyright.
Or what about using the likeness of a child on the body of a consenting adult for a deepfake? I that case nobody is harmed to begin with.
> Libel is a method of defamation expressed by print, writing, pictures, signs, effigies, or any communication embodied in physical form that is injurious to a person's reputation, exposes a person to public hatred, contempt or ridicule, or injures a person in his/her business or profession. [1]
A pornographic deepfake is certainly a "defamation expressed by [...] picture [...] that is injurious to a person's reputation" insofar as it creates a false impression about the person which is detrimental to their reputation.
[1]: https://www.thefreedictionary.com/libel
https://www.reddit.com/r/europe/comments/arhzub/pornstars_pe...
edit: not trying to be snarky, there are some good links above the fold.
https://www.pewresearch.org/global/interactives/global-moral...
The Czech Republic is the country most accepting of premarital sex with a 67% acceptance rate.
Simultaneously these porn companies are making exorbitant amounts of money from obviously illegal content. Something needs to be done but at the same time I worry this could be a political opportunity for a conservative to push forward an internet censorship bill.
People have not seen the Emma Watson porn, and those that have can probably read the word deepfake in the title. My guess is between 5% and 20% of the people who have seen it, and found it through searching for Emma Watson, rather than Emma Watson Deepfake believe that they have seen actual Emma Watson in porn.
But ultimately, you said that you didn't believe that it was irresponsible to assert that deepfakes will society more skeptical of video. That is your claim. I pose that deepfakes can easily be used to produce videos which are closer to what their audiences expect than reality, and as a result they will likely be less suspicious of what they see.
Even if factually deepfakes don’t substantially differ from impersonations, on a moral level they should not be tolerated, especially for grass roots female politicians.
Thanks for the Paylin reference
As is, it is already illegal to use Emma Watson's "likeness" without her approval in pretty much every western country, including the EU and the Czech republic. "likeness" here means that the fake is good enough for regular people to recognize the actress.
For example, see Midler vs. Ford: http://rightofpublicity.com/pdf/cases/midler.pdf
The article continues: “Until there is a strong reason for [porn websites] to try to take them down and to filter them, I strongly believe nothing is going to happen,” Patrini says.
Obviously. That "strong reason" is called a lawsuit and is pretty easy to organize. You might even sue for punitive damages and recoup your legal costs, meaning a celebrity successfully suing might be as expensive as $350,000 per video. I'd say that is a very strong reason to take things down, if - and apparently only if - the person requesting the takedown is following legal procedures.
"Under California law, intentional imitation of a celebrity’s distinctive and widely known voice for commercial purposes constitutes tortious misappropriation."
Intentional is the key word here. So I presume that if you look like Emma Watson and go with your real name, you're fine. If you pretend to be Emma Watson, then it becomes infringing.
But if the video says "this is a deepfake" and is not showing up in search results for her name, then it should be fine.
Its just that in the latter case, probably nobody is going to watch it, so the discussion is then moot anyway.