Most people think deepfake will be something massive like this. I suspect it's going to be very subtle. With a few words and phrases changed. Imagine a politician saying, "I'm for low taxes" and it's changed to "I'm for high taxes" Just one word that changes everything and has a big enough impact.
Or imagine it at scale. It seems there are two uses of deepfake which have separate effects:
1) High profile low volume content that trends (article)
2) Low profile high volume content spam (your point)
The second one seems like a much harder problem than the first (but both matter). Everyone already gets dozens of spam/phishing emails and links a day and not much has been done about it. I have no idea what the impact of deepfakes on the latter even looks like.
It isn't just the ability to change a message subtly and play both sides, it is the ability to deploy that at scale to highly siloed and divided audiences. This happens to be a known tactic of Russian active measures btw.
We've seen the impact of that politically, but I worry what will come during an actual large-scale military conflict when the battlefield very much becomes digital and attempting to subvert civilians to the cause.
As computing power grows these attacks will become something that can be performed in real-time on a single target. You could hack a CEO to introduce literal fake news that they would then act on, or change the instructions on a bomb-building video so that a "required ingredient" is exclusively sourced from a local store with direct contact with the FBI.
Did your friend on Skype really tell you to meet them on Main and Broadway at 4PM?
Or even more subtle. A tired looking politician says "I'm for, um, low taxes" and it's changed to a slightly more handsome version saying slightly more charismatically, "I'm for low taxes."
A single word won't work, if you have any context.
A thing which works to some degree is some more subtle editing (see the recent case of the slow Pelossi video) making people appear drunk or sick can be powerful.
The 2020 election is going to be interesting with deepfake tech getting better and better. Either everyone will be fooled, or everyone will quickly doubt any video ever presented. Either way, the candidates will need to find a new way to spread their propaganda (I mean campaign messaging) that people trust, if they stop trusting video.
When everyone can scream fake news we'll have to take politicians at their word. While President Trump was the first he won't be the last to use this tactic, and it may be a step towards debate turning to policy again rather than personal background gotchas etc.
> and it may be a step towards debate turning to policy again rather than personal background gotchas etc.
That would be a nice change! I think that's also the most optimist view. A pessimist would say that the end is no one trusts anything and we get more fractured and partisan.
1. Internet mobs will be de-clawed and any random person on the street that may have had their private information used against them now has instant plausible deniability. This is a big step towards valuing due process and personal privacy as digging into people's personal histories will become far less valuable than before if it can simple be handwaved away.
2. This could be a step back from the balkanization of the media. If anything can be faked than personal reputation and organizational reputation will be become far more valuable when it comes to the spread of stories.
Unfortunately, history proves this wrong. The average person on the street won’t have plausible deniability because deepfakes won’t be good enough to avoid detection. But rich and powerful people and governments will be able to create undetectable deepfakes to fool the public while denying such a thing is possible.
Regarding point 2, I thought that at first, but am now less certain. If fake stuff is everywhere, it's true. But if people then gravitate to reputable sources, there's less reason to make fake stuff, decreasing the amount of fake stuff until the incentive towards reputation is gone again. There's going to be an equilibrium in there somewhere, but that may be weirder and less predictable than either end of the spectrum.
So this is the future - we can't trust anything and so we trust nothing but our pre-existing biases
Bugger that.
This is a tech-created problem it can be tech-solved.
1. Photo manipulation leaves traces on the pixels, and this must be worse.
2. cameras can hash / sign every other frame or similar - we can build a chain of hardware to image that can be followed - and of an image or video does not have a CA cert chain it should be as mis-trusted as a web site without TLS
I have a friend who was working on #1 specifically. They got out of it after they realized it was just playing whack-a-mole.
This is honestly one of the very few use cases of cryptography signing a video or picture that I can see. But that would mean amateur journalists or the common person might be excluded from being a trusted source with video evidence.
Why? If my iphone camera signs every 24th frame (seems a conservative estimate of speed to do hash and sign), using a combination of the apple camera CA public key and my private key in the secure enclave than we can be reasonably sure that the raw footage now on my site came from a genuine iphone and from the phone I registered (somewhere)
All we are doing is trusting that the video images came from a given piece of hardware that we can trust belongs to me.
Whether I can be trusted is the same age old problem we have always had
But this simply makes it exponentially harder to fake or alter digitally any image (or at least look stupid when someone says "Citation Needed") - if you cannot produce the original raw footage questions will be asked why.
I can imagine this being something wanted on police bodycams quite soon
Killer blockchain app: Couldn't we just use blockchain to protect against video deepfakes? Hash every frame as the video is recording, then create an immutable ledger of frames. All "official" videos must be able to be verified with said ledger, and the ledger itself can only be verified with vendor-specific key licensed by a central authority. Perhaps two -- say, Apple has one for iPhone and they must verify as well.
This could certainly be the billion dollar opportunity blockchain fanatics have been looking for.
How would this prevent anything? A deepfake is essentially video which has been edited. Most newsstories have also been edited, either by processing the sound, video or adding fx and graphics. How would you verify these edits against the ledger, and how would they differ from a deepfake?
I'm not sure about the blockchain part but I would suggest that we are just trying to prove that there exists raw footage (is an hours interview from which the one minute segment was edited) and that footage was shot on a unique identifiable camera, whose history can be traced.
If the video of politician X making unguarded remarks comes from a camera owned by CNN for the past two years it is a different level of trust than it coming from one bought last week with a credit card in the name of politicians opponents new intern.
> 2. cameras can hash / sign every other frame or similar - we can build a chain of hardware to image that can be followed - and of an image or video does not have a CA cert chain it should be as mis-trusted as a web site without TLS
Why would this not be easily beatable? Hardware hacking aside, why not the naive approach of filming a monitor playing a deepfake?
The important part there would be chain of trust. You should be able to see, given a video, "oh, that was recorded by some guy who lives in a basement; I can figure out he was never even /near/ Washington" (or "that doesn't have a signature. What is this, 2019?!") and treat it accordingly. It means you need to give up your own identity if you want to publicly verify that a video you've captured is yours, but that's kind of the point. Anonymity is great, but it isn't a source of trustworthy information. Never has been, never will be, and now we have the math to say it.
So basically GPG for cameras, but with a hip startup name? Maybe any camera would ship with its own private key out of the box, and there'd need to be some kind of key rotation as well as a way to push your public keys … somewhere(s). I guess an attack here is somebody could lift a private key from someone else's camera and use that key to sign their fakes, but at least that's a reasonably tangible sort of attack that people know how to protect against.
(And of course we'll never stop people from watching 24/7 news every minute of the day, so we're all doomed anyway).
But at most you can prove which camera shot a picture, and its original owner... Kind of, because there's nothing stopping a photo agency from signing the pictures after they're taken, in order to anonymize their contributors.
So, if I want to fake a picture and make it a bit more convincing, I only need to get my hands on a few cameras from pawn shops in Washington. I just can't say "this is a Reuters photo" or "this is a banned commercial" or whatever.
> 2. cameras can hash / sign every other frame or similar - we can build a chain of hardware to image that can be followed - and of an image or video does not have a CA cert chain it should be as mis-trusted as a web site without TLS
Sure! Also, by total coincidence, now it's possible to trace every single photograph ever taken to whoever took it. A bunch of photojournalists in Syria, Iran, Mexico, and Missouri go missing, but I'm sure that's a coincidence.
Not if the hashes are also hidden as pixel structure in the image. Something already done for preventing screenshot leaks (World of Warcraft iirc) or to trace movie leaks.
Does the information hidden inside a screenshot from WoW is resilient enough to survive a capture from a cell phone camera with JPEG compression and everything in between?
What you’re suggesting is essentially DRM for news, where only an approved subset of sources would be blessed as non-fake. Might as well go back to the days of 3 TV channels.
I agree that it's eminently tech-solvable. Experts can still analyze the pixels and determine if anything fishy happened. I think the societal problem is more if/when people make up their minds to not trust those experts when they call out a fake and just believe it anyway because they want to believe it.
An example would be, some fake video circulates of some politician saying or doing something they never said or did. Maybe someone created it to harm that politician, maybe it was a prank, doesn't matter. A day later, a gazillion experts do the analysis and come out with public evidence that the video was a fake. Great. Then someone like Alex Jones tells his millions of listeners that he knows better, and some world-class expert who he mysteriously won't name told him privately that the video was real, and that somehow all of those experts were in on a conspiracy to debunk it, even though the evidence they provided debunking the video is all out in public (if perhaps too technical for most people). And then, voila. Millions of people now decide that it was real after all, and that all of those scientists and engineers and law enforcement people who said it was fake are all in on some conspiracy to squash the truth.
So you're right, it is a tech-created problem, and presumably tech solutions will always exist, but as long as you have an uneducated segment of the population that will always just believe whatever their tribe tells them, those tech solutions won't even matter. I don't know how to solve that problem, other than people becoming more educated and less geographically isolated, so that they can recognize and trust expert analysis when it's available and trustworthy.
Put another way, the problem isn't that Alex Jones exists, but that there exist people who think he's credible. The very existence of his listeners is a much bigger problem than his own existence. There will always be conspiracy theorists and there will always be faked evidence to support them, but what holds society together is our collective ability to acknowledge well-documented bullshit as bullshit, regardless of what tribe someone thinks they belong to and regardless of what they want to believe.
In the fifties there were fake photos of UFOs.. as well as ghosts earlier than that. Whether it's an image or a video it can be faked and should not be absolute proof of anything
Have you seen how they use projections in practical effects? What stopping me from casting the deepfake into the plain of a camera and getting a valid and signed footage?
I don’t think you understand just how problematic it is to verify anything in the analog world.
Say you sit at a bar and someone displays a news clip on the TV? How do you know it’s real? Someone video or voice calls you with a deepfake of a loved one are you going to do forensics on it before acting on what they asked you to do?
I'm going to make a moderate to strong-ish claim that deepfakes will not be a substantially serious issue for Democracy or reporting. I'm not certain, but I feel reasonably confident.
- We've already had sophisticated image editing capabilities for a long time, and it hasn't substantially changed how newspapers, blogs, and social media accounts share photos. In particular, I see screencaps of Twitter threads all the time, even though they're trivial to fake. For whatever reason, people already seem to be OK with this. Maybe image editing did upset everything and I'm just too young to remember, but in either case, it doesn't look to me like the world fell apart.
- We've had (relatively) sophisticated audio editing capabilities for a somewhat lesser period of time, and that hasn't changed the landscape of how I see audio reported on. Reporters still use audio as evidence in articles, we haven't seen a wide spread of proprietary chains of trust, anything like that.
- We've seen decent evidence that deepfakes aren't required to currently fake video content to the point that you can deceive people. Deceptive cutting, slowdown, etc... seems to be enough. For people who want reasons to distrust a video, those methods also seem to be adequate. I regularly encounter people who'll look at body-cam footage and say, "well, you don't know, it doesn't show the full context." I'm skeptical deepfakes will make this situation worse -- at most I suspect they'll just further divide us into groups who research sources and groups who don't.
- Deepfakes are getting better, and I keep on getting told that they're eventually going to be perfect, but again, in practice I can't help but compare it to Photoshop. It turned out that with Photoshop, people's ability to spot fakes improved at roughly the same rate as the technology, to the point where I don't think it would be trivial for me today to make a fake photo of a politician that wouldn't get quickly identified as such on Reddit or Twitter. I've seen some really impressive AI generated content that blows me away, but right now I can still tell that it's all fake. I haven't seen anything that can consistently produce results that fool me beyond the most straightforward, boring use-cases. And I'm not an expert on this stuff, if it's good it should be able to fool me.
In theory, deepfakes are concerning, and I get why people are concerned about them. The technology will get better, it will be easier and easier for anyone to make a video saying anything they want. In theory, I agree with all of that. I just don't see a lot of practical evidence that it's going to be any better or worse than what we have right now.
If I hadn't grown up in an era where people were making what to me sounded like similar claims about Photoshop, I would be more worried.
most everyone already doesnt consider anything they dont already believe. everyone will just shout fakenews at everything real unless they want to believe.
any trust left in institutions will erode. we cant even believe our eyes or ears.
But what's different? Why is this a different concern now than it was 5 years ago?
Photographic evidence hasn't stopped Trump from believing his inauguration was larger than Obama's. And it has always been easy to get people en-mass to believe hoaxes using doctored evidence.[0][1]
I think I agree. People already take political "memes" (some text on an image, designed to provoke reaction) for granted, and still-images are easier/faster to consume, and create, than videos.
I agree with this sentiment. However, there's another threat from deepfakes that someone had to point out to me: generating false memories.
Imagine you're a VIP and someone shows you footage of yourself doing something unacceptable from 30 years ago. There's a good chance that you might believe that footage and accept that as a true memory, even though it is entirely fake.
Would that scenario be significantly harder to pull off today with a faked audio file or photograph?
I'm not necessarily being theoretical about that. Growing up, I used to Photoshop family Christmas photos when not all of us could get together at the same time. Near the end of their lives, I vaguely remember it causing my grandparents to occasionally get confused about who was or wasn't actually at a gathering.
And it's not like I did a particularly great job of it -- they just didn't have experience with what the technology was capable of.
I guess what I'm trying to say is, yes, it seems obvious that there will be a generation of people who are not equipped to deal with the current state of image/audio/video manipulation, but is that actually a new thing? Hasn't that kind of always been the case?
I kind of suspect that fake videos will not be much more of an issue than photoshopped images. I don't recall with certainty but I believe there was some fear surrounding the ease at which photographs could be manipulated in the digital age. Some combination of a general growing awareness of the ease at which photos could be altered along with continued advancements in the ability to determine the authenticity of photographs has rendered faking of photographs relatively harmless. In fact, I think the feedback loop has resulted in not being a huge market for faking images at least for political gain. I am not referring to photoshopping images of models and things to just make them look more attractive. Also, although it is very easy to doctor an image, in general people have still managed to keep a level of intuition which allows them to not suspect every image they see as being fake.
I've been following the press about deep fakes for a while now, and it's been full-on negative, gloom and doom, a potential threat to every democracy on the planet, so on and so forth, in that fashion.
I suspect this is a smoke screen for something coming. There will be an event where a very real video emerges, likely of a political elite, and its contents will be so horrific and damming that the only defense will be to claim that it is a deep fake.
A large chunk of the public, now well primed for this, will accept the argument, even though many people will go to great lengths to show how the video is actually authentic.
But the jury has been poisoned already. The waters are muddied. There will be no conversation, and the mainstream, already pumping out story after story about how awful these deep fakes are, will continue to take up this position, and the sleeping masses will continue to sleep.
Keep your wits about you. The truth wants to come out.
The Rules of Evidence in Federal Court (and the States) make it fairly easy to introduce photo/video/audio. This needs to change. The rules regarding admissibility of this type of evidence were drafted when fakes were difficult.
The treatment of emails, however, does not inspire much hope. That is, in most courts, a printout of an email is treated -- as a practical matter -- similarly to a document signed by the author. Very frustrating. There is nothing magical about a piece of paper with an email header... you can create same in two minutes with a word processor. And don't even
Get me started with text messages... try to introduce an unsigned typed scrap of paper and you'll get laughed out of court; but, put the text into something that looks like a screenshot of a phone and, voila, instantly admissible as an authentic text message.
50 comments
[ 4.5 ms ] story [ 109 ms ] thread1) High profile low volume content that trends (article)
2) Low profile high volume content spam (your point)
The second one seems like a much harder problem than the first (but both matter). Everyone already gets dozens of spam/phishing emails and links a day and not much has been done about it. I have no idea what the impact of deepfakes on the latter even looks like.
We've seen the impact of that politically, but I worry what will come during an actual large-scale military conflict when the battlefield very much becomes digital and attempting to subvert civilians to the cause.
Did your friend on Skype really tell you to meet them on Main and Broadway at 4PM?
A thing which works to some degree is some more subtle editing (see the recent case of the slow Pelossi video) making people appear drunk or sick can be powerful.
That would be a nice change! I think that's also the most optimist view. A pessimist would say that the end is no one trusts anything and we get more fractured and partisan.
1. Internet mobs will be de-clawed and any random person on the street that may have had their private information used against them now has instant plausible deniability. This is a big step towards valuing due process and personal privacy as digging into people's personal histories will become far less valuable than before if it can simple be handwaved away.
2. This could be a step back from the balkanization of the media. If anything can be faked than personal reputation and organizational reputation will be become far more valuable when it comes to the spread of stories.
Bugger that.
This is a tech-created problem it can be tech-solved.
1. Photo manipulation leaves traces on the pixels, and this must be worse.
2. cameras can hash / sign every other frame or similar - we can build a chain of hardware to image that can be followed - and of an image or video does not have a CA cert chain it should be as mis-trusted as a web site without TLS
3. Err - there must be other ways.
This is honestly one of the very few use cases of cryptography signing a video or picture that I can see. But that would mean amateur journalists or the common person might be excluded from being a trusted source with video evidence.
All we are doing is trusting that the video images came from a given piece of hardware that we can trust belongs to me.
Whether I can be trusted is the same age old problem we have always had
But this simply makes it exponentially harder to fake or alter digitally any image (or at least look stupid when someone says "Citation Needed") - if you cannot produce the original raw footage questions will be asked why.
I can imagine this being something wanted on police bodycams quite soon
Killer blockchain app: Couldn't we just use blockchain to protect against video deepfakes? Hash every frame as the video is recording, then create an immutable ledger of frames. All "official" videos must be able to be verified with said ledger, and the ledger itself can only be verified with vendor-specific key licensed by a central authority. Perhaps two -- say, Apple has one for iPhone and they must verify as well.
This could certainly be the billion dollar opportunity blockchain fanatics have been looking for.
If the video of politician X making unguarded remarks comes from a camera owned by CNN for the past two years it is a different level of trust than it coming from one bought last week with a credit card in the name of politicians opponents new intern.
Why would this not be easily beatable? Hardware hacking aside, why not the naive approach of filming a monitor playing a deepfake?
So basically GPG for cameras, but with a hip startup name? Maybe any camera would ship with its own private key out of the box, and there'd need to be some kind of key rotation as well as a way to push your public keys … somewhere(s). I guess an attack here is somebody could lift a private key from someone else's camera and use that key to sign their fakes, but at least that's a reasonably tangible sort of attack that people know how to protect against.
(And of course we'll never stop people from watching 24/7 news every minute of the day, so we're all doomed anyway).
So, if I want to fake a picture and make it a bit more convincing, I only need to get my hands on a few cameras from pawn shops in Washington. I just can't say "this is a Reuters photo" or "this is a banned commercial" or whatever.
Sure! Also, by total coincidence, now it's possible to trace every single photograph ever taken to whoever took it. A bunch of photojournalists in Syria, Iran, Mexico, and Missouri go missing, but I'm sure that's a coincidence.
Traceability is not without its drawbacks.
Not perfectly, but watermarking images in ways that survive re-encoding is far from impossible or unknown.
Oh man, you just articulated something nagging at me about deepfakes that I hadn't been able to.
An example would be, some fake video circulates of some politician saying or doing something they never said or did. Maybe someone created it to harm that politician, maybe it was a prank, doesn't matter. A day later, a gazillion experts do the analysis and come out with public evidence that the video was a fake. Great. Then someone like Alex Jones tells his millions of listeners that he knows better, and some world-class expert who he mysteriously won't name told him privately that the video was real, and that somehow all of those experts were in on a conspiracy to debunk it, even though the evidence they provided debunking the video is all out in public (if perhaps too technical for most people). And then, voila. Millions of people now decide that it was real after all, and that all of those scientists and engineers and law enforcement people who said it was fake are all in on some conspiracy to squash the truth.
So you're right, it is a tech-created problem, and presumably tech solutions will always exist, but as long as you have an uneducated segment of the population that will always just believe whatever their tribe tells them, those tech solutions won't even matter. I don't know how to solve that problem, other than people becoming more educated and less geographically isolated, so that they can recognize and trust expert analysis when it's available and trustworthy.
Put another way, the problem isn't that Alex Jones exists, but that there exist people who think he's credible. The very existence of his listeners is a much bigger problem than his own existence. There will always be conspiracy theorists and there will always be faked evidence to support them, but what holds society together is our collective ability to acknowledge well-documented bullshit as bullshit, regardless of what tribe someone thinks they belong to and regardless of what they want to believe.
I don’t think you understand just how problematic it is to verify anything in the analog world.
Say you sit at a bar and someone displays a news clip on the TV? How do you know it’s real? Someone video or voice calls you with a deepfake of a loved one are you going to do forensics on it before acting on what they asked you to do?
- We've already had sophisticated image editing capabilities for a long time, and it hasn't substantially changed how newspapers, blogs, and social media accounts share photos. In particular, I see screencaps of Twitter threads all the time, even though they're trivial to fake. For whatever reason, people already seem to be OK with this. Maybe image editing did upset everything and I'm just too young to remember, but in either case, it doesn't look to me like the world fell apart.
- We've had (relatively) sophisticated audio editing capabilities for a somewhat lesser period of time, and that hasn't changed the landscape of how I see audio reported on. Reporters still use audio as evidence in articles, we haven't seen a wide spread of proprietary chains of trust, anything like that.
- We've seen decent evidence that deepfakes aren't required to currently fake video content to the point that you can deceive people. Deceptive cutting, slowdown, etc... seems to be enough. For people who want reasons to distrust a video, those methods also seem to be adequate. I regularly encounter people who'll look at body-cam footage and say, "well, you don't know, it doesn't show the full context." I'm skeptical deepfakes will make this situation worse -- at most I suspect they'll just further divide us into groups who research sources and groups who don't.
- Deepfakes are getting better, and I keep on getting told that they're eventually going to be perfect, but again, in practice I can't help but compare it to Photoshop. It turned out that with Photoshop, people's ability to spot fakes improved at roughly the same rate as the technology, to the point where I don't think it would be trivial for me today to make a fake photo of a politician that wouldn't get quickly identified as such on Reddit or Twitter. I've seen some really impressive AI generated content that blows me away, but right now I can still tell that it's all fake. I haven't seen anything that can consistently produce results that fool me beyond the most straightforward, boring use-cases. And I'm not an expert on this stuff, if it's good it should be able to fool me.
In theory, deepfakes are concerning, and I get why people are concerned about them. The technology will get better, it will be easier and easier for anyone to make a video saying anything they want. In theory, I agree with all of that. I just don't see a lot of practical evidence that it's going to be any better or worse than what we have right now.
If I hadn't grown up in an era where people were making what to me sounded like similar claims about Photoshop, I would be more worried.
any trust left in institutions will erode. we cant even believe our eyes or ears.
Photographic evidence hasn't stopped Trump from believing his inauguration was larger than Obama's. And it has always been easy to get people en-mass to believe hoaxes using doctored evidence.[0][1]
[0]: https://en.wikipedia.org/wiki/Cottingley_Fairies
[1]: https://www.snopes.com/fact-check/dead-fairy/
Imagine you're a VIP and someone shows you footage of yourself doing something unacceptable from 30 years ago. There's a good chance that you might believe that footage and accept that as a true memory, even though it is entirely fake.
I'm not necessarily being theoretical about that. Growing up, I used to Photoshop family Christmas photos when not all of us could get together at the same time. Near the end of their lives, I vaguely remember it causing my grandparents to occasionally get confused about who was or wasn't actually at a gathering.
And it's not like I did a particularly great job of it -- they just didn't have experience with what the technology was capable of.
I guess what I'm trying to say is, yes, it seems obvious that there will be a generation of people who are not equipped to deal with the current state of image/audio/video manipulation, but is that actually a new thing? Hasn't that kind of always been the case?
I suspect this is a smoke screen for something coming. There will be an event where a very real video emerges, likely of a political elite, and its contents will be so horrific and damming that the only defense will be to claim that it is a deep fake.
A large chunk of the public, now well primed for this, will accept the argument, even though many people will go to great lengths to show how the video is actually authentic.
But the jury has been poisoned already. The waters are muddied. There will be no conversation, and the mainstream, already pumping out story after story about how awful these deep fakes are, will continue to take up this position, and the sleeping masses will continue to sleep.
Keep your wits about you. The truth wants to come out.
The treatment of emails, however, does not inspire much hope. That is, in most courts, a printout of an email is treated -- as a practical matter -- similarly to a document signed by the author. Very frustrating. There is nothing magical about a piece of paper with an email header... you can create same in two minutes with a word processor. And don't even Get me started with text messages... try to introduce an unsigned typed scrap of paper and you'll get laughed out of court; but, put the text into something that looks like a screenshot of a phone and, voila, instantly admissible as an authentic text message.