This is too easy, I struggled a bit with no sound but video to detect real or fake because the lighting can be a bit weird but for the voices you can hear it in a split second that they're fake
Agreed. The video and text fakes are getting better, but the audio (at least here) isn't close. It's not even an uncanny valley, it's worse than an SNL parody.
In cases without audio, watching the lips for sync was also a reliable giveaway.
I only did about half of the samples, but things became harder further in. I'm not sure if they were better done or I was just becoming bored and losing focus though.
I didn't do the test. My first thought was that it's the wrong question. A "trusted sources" model for believing stuff can always be gamed. There should be more focus on independent confirmation, and general sanity checking, rather than caring about whether a video is fake or not, especially if one is going to take some action based on the content.
Also, the fake ones with audio or text from Trump are way too easy for anyone who has seen Trump over the years to guess (as opposed to those who know Trump from "comedy news shows" from the last few years)... because Trump wouldn't say some of those things (anti-gay marriage stuff, for example).
Biden is a bit more difficult to guess like that because he could have said anything which he thought would be popular at a given point throughout his career... but then the technology is too bad to actually be convincing enough.
Mildly surprised the quality of this project coming out of MIT. Not only the UI was quite hard to use, I could not get beyond 10 samples. They were annoyingly obvious. Why would they not choose to not to make this as tricky as possible?
It’s clear that state of the art deep fake technology can do better than what’s presented here.
What if the whole point of this exercise is to instill (false) confidence in the most people that they have the ability to distinguish a deep fake from the real thing?
For the record, I don’t believe this is the case and I fully expect that subsequent iterations of this exercise will be far more difficult.
I really wish they took actual things the individual said in the past, but used a deepfake for them actually saying it. Most of the deepfakes were easy to spot, but a couple weren't clear at first that they were fake, only to be given away by the content of their speech.
It doesn't necessarily have to be. For example what if you modified a clip of a political candidate asking for funding and only modified the name of the organization/website their supporters should go to in order to donate.
Getting the voice accurate would be very hard, even if you only used existing words they've said to make up the false domain. I've tried editing audio that way and it still sounds like when a video game ai stitches dialog together.
Getting the voice accurate is actually really easy, and is done all the time. https://en.wikipedia.org/wiki/Overdubbing is used in music and movie production, and you can change out a few words with "ease" [1]. You use an impressions actor, get them to say the whole line, matching the cadence, and you've got a decent amount of skew in mouth movement because people aren't always lipreading carefully.
None of these techniques are out of the budget of a college film or dedicated hobbyist.
Overdubbing is far easier with an instrument than with a voice, the article doesn't even list examples of an impressionist over dubbing someone, just overdubbed duets. Also what you're describing is not overdubbing, it's ADR. Overdubbing is just that, dubbing over the existing audio. Not only would you need to mimic the speech but you'd have to recreate the ambient sound as well. Even with a person speaking into a microphone in a silent room the change in ambience is noticeable unless the whole scene is using a false ambience track separate from the voice track.
And I disagree with you about the lipreading. There's a reason ADR in films is always done with a shot of the back of the actors head.
Even when it's done with literally the same original speaker it's pretty obvious. Or have you never fooled a stranger in the alps?
But the person saying something outrageous would undermine the point of the study--to determine if deep fakes were indistinguishable from actual footage. If it's given away by something unrelated to the deepfake method, then you're adding in an external bias unrelated to the realism of the deepfake.
Unless I misunderstood the goals of the study of course.
The video look worse than "professional fakes" long even before AI.Text is the hardest for people who don't follow those specific people(though it is still easy to spot the fake), and audio is very easy aswell.
Not boasting, just reality.I've seen some pretty astounding deepfakes, these are not part of those.
The video fakes are always blurry around the mouth and gestures don't match the words.
The audio fakes are too "clean" (too defined breaks between words and no background noise) and sound like caricatures.
Text is virtually impossible because politicians say "fake" stuff all the time (e.g. speeches someone else wrote) so you can't rely on uncharacteristic language.
I only got one of the text ones wrong. I think content was my key indicator. The fakes seemed more like what the opposite party would try to make them say... like a caricature of whatever message they actually said.
The only mistakes I made were calling real ones fake, but I was always able to detect the fakes. Some of the audio sounded too clean but turned out to be real.
A much easier criterium: the real speakers were speaking continuous sentences with logical spots for breathing pauses, the fakes discrete words. And I guess that's also why I had a harder time with the silent clips, then you don't have that signal.
It would be interesting to test how well any of us could spot these fakes if we weren't already on a web site priming us to look for them. How much of the skepticism is because you are on "alert" vs one's baseline levels of skepticism.
To me they always look like uncanny valley so my brain flags them as fishy. Everything deepfake feels like watching Gemini Man to me.
But then again, I am pretty paranoid as I do think the media and web are not out there to provide me with quality; they want to achieve money and/or power and will try to manipulate me in order to do that so I would assume fake in case of doubt.
> The audio fakes are too "clean" (too defined breaks between words and no background noise) and sound like caricatures.
The lack of background noise alone was a pretty instant key for me. And adding realistic background noise, appropriate echo/reverb that would match (even roughly as one-on-one conversation vs podium at press briefing) the room they are speaking in aren't even things you need deepfake for and would be simple audio post-processing.
The impersonators were so bad that I assumed they were intentionally added as a control. Anyone who called those real should likely be discarded from the final results.
Yeah, the fake audio clips had way too many telltale signs of a fake to be plausible. One of the Trump ones sounded like an SNL skit attempting his accent.
The silent videos are a pointless test because it could easily be poor quality video, and what does fake mean if it’s a video of the politician speaking behind the podium?
The videos with spoken words were very easy to spot, and the ones that dealt with substantive content you could tell by the intonations.
You can still see some bizarre artifacts in the silent fakes. Biden’s teeth moving in a different direction than the rest of his jaw is an obvious tell.
Exactly my feeling about this. Biden and Trump also both ramble incoherent sentences all the time, so they do talk like GPT-3, making it much harder to discern text fakes here.
The only thing that made me score higher than random on the text segments was assuming that the stereotypical extreme things for Trump and Biden to say are usually fake.
I.e. they used a bit of truth but stretched it too far.
This is just nihilistic cynicism. Both Trump and Biden speak in easily understood sentences. I would tend to disagree with Trump more often than not, but that doesn't mean he isn't a clear communicator. Biden is different in that he choses his words more carefully. There are many questions where it is better for him and the US to not answer a question, so he won't.
But nothing about either person's sentences is 'incoherent'. And 'stretching the truth too far' is something entirely different. Incoherent sentences, by definition, cannot be 'stretching the truth', because they convey no meaning.
I'm not saying everything that Trump and Biden say is like this, but the existence of so many bumbling speech examples from both makes it really hard to discern what is real and what is fake.
I saw one of Donald and was like ha ha, no way, those ears are obviously AI-generated. It was real.
There’s a great Just William story where he sneaks into a waxworks exhibition and pretends to be one, and a young man comes along and points out all the defects in his utterly unrealistic anatomy. It was a bit like that.
I found the text easy because my understanding of what they would say is based on their positions in reality than the fictional caricatures of them in the opposition party blogs.
If you weren't familiar then I could see it being hard.
To label the practice of speechwriting as 'fake' is deeply misguided. Is the contract you signed 'fake' because you didn't write it? Do you believe politicians should be judged primarily on their writing skills?
A politician choses their speechwriters, works with them, and takes ultimate responsibility. The process just happens to be exactly the same as it for all other tasks they do. I would argue a demonstrated ability to choose good people and judge their work is so essential for high government office, the process involving speechwriters is a far better measure of their suitability for office.
Speechwriters, at least good ones, also don't write 'generic' texts. They will study their client's speech patterns and adjust word choices, rhythm, humor, and other patterns to fit the person's style. For offices as high as the US president, the writers tend not to change, sometimes over decades.
For context: I'm mostly unaware of actions of US presidents (beyond the broad sweeps based on left vs right) and I've at most listened to Trump and Biden maybe a couple of times for a total of ~1min in the last few years - just doing this exercise has probably at least doubled my exposure.
I found the text and voice mostly impossible unless the content clued me in. Video was a little easier because I know to look for teeth, video with sound was fairly easy.
Is there a generic trick for recognising faked voices without really knowing the original (similar to looking for teeth on videos)?
For the speech clips I thought the disjointedness of the words made it obvious. That being said, Trump has a very recognizable way of talking so his clips were really easy to sort.
Same here! I'm not a native English speaker, I live in the US but don't follow politics. I got a lot of Biden ones wrong because I haven't watched as many videos of him as Trump's videos. I also marked some real videos as fakes because I thought there is no way Biden or Trump would talk about this specific subject in this way.
Maybe we are missing some tricks to pick out the fakes.
For the majority of folks here who picked out the fakes easily, do you feel if you can do the same if the videos were of Putin speaking in Russian (with subtitles), or speaking in English with a very thick accent?
There are real world effects in the real videos that are completely absent in the fake ones that give them away.
The fakes have completely clean audio with similar artificial noise (real ones have crowd, or other backround noise), the fakes have no microphone or environmental artifacts (echo/reverb, plosives, others).
What made them even easier to recognize is that accents were all wrong. They were exactly as another person said, caricatures of the original voices. I actually found that they were so obvious that most could be identified before the second or two that they make you wait to click the submit button.
If it were a foreign language, I think the success rate would be lower, but the unnatural lack of noise gives them away.
My guess is that they don't want to distort results. Any kind of scoring would present an incentive to start cheating (e.g. by not checking the "I've seen this before"-box), be it consciously or subconsciously.
From the presented samples I'd wager that one goal of the study might be to quantify the influence of audio-only vs video-only vs text, hence text-only, video-only, audio-only, and video with subtitles.
While text limits the decision making process to preconceived notions about the person in question and prior background knowledge, video and audio may present clear cues from human perception alone.
Been wondering about this in context of UKR and the huge role of open source intel via mobile phone videos in it. For understandable reasons it's not part of the discussion right now but there might be interesting retrospectives about this available down the line.
This was an unsatisfying experience. The visual and audio fakes are easy to spot, while the texts are short enough and correct enough that you don't have enough to go with in making a technical decision. I did however get them right by trying to figure out if the statement made sense in the context of the respective politician's general stance on the issue discussed.
Update:
Come to think of it, IRL text deep fakes don't even make sense. You can make up whatever quotes you like, you don't need GPT3 to do it for you.
> Come to think of it, IRL text deep fakes don't even make sense. You can make up whatever quotes you like, you don't need GPT3 to do it for you.
Exactly. John Oliver pointed out a few weeks ago that the Russian media was using stock footage of other nation's military actions (Finland?) for propaganda. No need for deep fakes. The lowest effort lie can still work.
Deep fake tech is the new photoshop. It's mostly used for good (art, memes, jokes with friends, Disney films).
You can imagine a future where deep fakes are utterly indistinguishable from real life, but I think that people will learn to recalibrate themselves against the new technologies. It may even cause us to think more critically: people frequently chime in, "how is this not photoshopped?" on social media. They'll call out videos of explosions and physics as "looking fake". They'll have the same response to purported deep faked videos.
All the same, I'm glad and appreciate that academic and policy folks are taking a close look at the technology. We need frameworks, ethics, and institutional familiarity.
> Come to think of it, IRL text deep fakes don't even make sense. You can make up whatever quotes you like, you don't need GPT3 to do it for you.
They make sense for advertising and misinformation purposes.
You don't have to pay a firm to drum up interest for a brand/product, theoretically, you'd be able to generate convincing conversations about the brand between bots on Reddit or Twitter, and do it at scale. Instead of paid posts on social media, you cut out the middle man and give the perception of community support for a brand/product, despite it being artificial.
The last point goes for misinformation, as well. Want to start grassroots support for an idea/policy/person/group/place? Instead of paying teams of people to post on social media about it, you can automate the campaign and scale it with deep faked text.
I don't think deep fakes are there yet for either purpose, though, and it isn't a given that they'll be good enough for them, either.
I'm pretty certain that in at least one of these Trump is standing in front of a green screen, I put fabricated, but I guess that was wrong to do because that wasn't fabricated with AI? Either way this said it was real, which yeah it was, but the setting was faked. Idk.
Anyway, THANK YOU site developers for properly handling resumption of progress and not abusing the history API. Greatly appreciated and unfortunately very rare for these sorts of sites.
I only did 12 of them before exiting but got 100% of them. I watched them on my phone without turning it landscape to simulate how I'd normally watch a brief news clip like this.
They're still not particularly convincing, especially the audio ones sound very fake.
But I think these sorts of tests are flawed. The fact that it's a test primes you to look for these details in a way you might not if these clips were shared on Twitter/Reddit/etc. The deepfakes are still pretty obviously fake, so I don't think I'd fall for one "in the wild". But I also see this style of test used for things like "can you hear the difference between these 2 audio compression formats" or "does this virtual guitar amp sound just like the real thing", and that's just not how you consume real media. For example I'd probably never know Meshuggah switched to entirely digital guitar amps years ago if it weren't for them saying it in interviews.
When deepfakes one day do get as close to the real thing as digital guitar amps have gotten to real amps, I doubt I'd notice a random r/worldnews post is faked, even if I could still reliably pass an A/B test.
This is the real test. While we (HackerNews) readers might be able to realize they are fake, that other 90% of the world won't.
The reverse side of it is if an overwhelming amount of deep fakes get called out, people will stop trusting the real ones (we have already started stepping into that reality)
Perhaps you detected them because you knew there would be 50% fakes? If you'd see one of these fakes in a CNN article, would you still reliably detect it? I think the role of the prior expectation should not be underestimated.
I think it's one of those studies where they say they are testing for X but they are actually testing for Y. Not sure what Y could be. But I don't even live in the US or consume it's media, and could still figure out all of the examples I saw/listened.
The test seems flawed. I got the first one (after the attention check) wrong - I said it was fake, but it was real. I thought it was fake because Trump's hair was clearly very smeared out, not at all realistic. But it seems that this must have been an artifact of heavy video compression. I don't see how this is a meaningful test of anything.
Yeah, but how is the test taker supposed to know whether to take the smearing as indicating it's fake or not?
Better would be to let them see 10 videos all at once, half of which are fake, and ask them to divide into a fake set of 5 and a real set of 5, after looking at all of them as many times as they like. Asking "fake or real" when there is no basis to tell whether flaws should be taken as indicating "fake" or just attributed to compression seems meaningless.
Or tell people what aspects of fakeness they're trying to assess - eg, forget about video artifacts, just pay attention to the audio.
Using clips of Trump and Biden is also a bad idea. They ask you to say if you've seen one before, but aren't many people going to have seen one, but not clearly remember that, and then be influenced to think it's real by sub-conscious recognition?
Why not present pairs of videos of the same non-famous person, one fake one real, both presented with the same amount of compression, and ask one to chose which is the real one? Using many different people, of course - why would you introduce doubt about the generality of your results by using only two people?
Of course, in practice people may be less able to recognize fakes when video quality is poor, which would be useful to know, but I think one would need to investigate that issue separately, not in combination with other reasons that fakes might or might not be recognizable.
I think starting with a guy who looks almost exactly like Tom Cruise in the first place really gives the computer the advantage it needs to look believable. You can see him here
https://youtu.be/p7-B8S734T4?t=77
144 comments
[ 3.9 ms ] story [ 201 ms ] threadIn cases without audio, watching the lips for sync was also a reliable giveaway.
Also, the fake ones with audio or text from Trump are way too easy for anyone who has seen Trump over the years to guess (as opposed to those who know Trump from "comedy news shows" from the last few years)... because Trump wouldn't say some of those things (anti-gay marriage stuff, for example).
Biden is a bit more difficult to guess like that because he could have said anything which he thought would be popular at a given point throughout his career... but then the technology is too bad to actually be convincing enough.
It’s clear that state of the art deep fake technology can do better than what’s presented here.
What if the whole point of this exercise is to instill (false) confidence in the most people that they have the ability to distinguish a deep fake from the real thing?
For the record, I don’t believe this is the case and I fully expect that subsequent iterations of this exercise will be far more difficult.
None of these techniques are out of the budget of a college film or dedicated hobbyist.
[1] https://www.youtube.com/watch?v=z4t6zNZ-b0A
And I disagree with you about the lipreading. There's a reason ADR in films is always done with a shot of the back of the actors head.
Even when it's done with literally the same original speaker it's pretty obvious. Or have you never fooled a stranger in the alps?
https://youtu.be/LCcKBcZzGdA
Unless I misunderstood the goals of the study of course.
Not boasting, just reality.I've seen some pretty astounding deepfakes, these are not part of those.
The video fakes are always blurry around the mouth and gestures don't match the words.
The audio fakes are too "clean" (too defined breaks between words and no background noise) and sound like caricatures.
Text is virtually impossible because politicians say "fake" stuff all the time (e.g. speeches someone else wrote) so you can't rely on uncharacteristic language.
I guess I'm just extra skeptical.
I did much better on them once I realized that the real speakers were much more likely to look around the room, or especially down toward the podium.
But then again, I am pretty paranoid as I do think the media and web are not out there to provide me with quality; they want to achieve money and/or power and will try to manipulate me in order to do that so I would assume fake in case of doubt.
The lack of background noise alone was a pretty instant key for me. And adding realistic background noise, appropriate echo/reverb that would match (even roughly as one-on-one conversation vs podium at press briefing) the room they are speaking in aren't even things you need deepfake for and would be simple audio post-processing.
Still the study is extremely valuable at probing the current state of things! I'm super happy they did this.
The videos with spoken words were very easy to spot, and the ones that dealt with substantive content you could tell by the intonations.
The only thing that made me score higher than random on the text segments was assuming that the stereotypical extreme things for Trump and Biden to say are usually fake.
I.e. they used a bit of truth but stretched it too far.
But nothing about either person's sentences is 'incoherent'. And 'stretching the truth too far' is something entirely different. Incoherent sentences, by definition, cannot be 'stretching the truth', because they convey no meaning.
They do, unless they don't. And that's the issue, because these text snippets could be a gaffe, or they could be GPT-3.
GPT-3 often gives you similar, somewhat meaningful sounding sentences that are a semantic mess.
https://youtu.be/4MyLwAokINc https://youtu.be/sli2Vn6Iiy4
I'm not saying everything that Trump and Biden say is like this, but the existence of so many bumbling speech examples from both makes it really hard to discern what is real and what is fake.
only one of these two does it with 100% consistency, especially if there's no teleprompter, even then he flubs it.
One wrote mostly coherent thoughts in 140 characters, the other has handlers. Guess which one got cancelled?
There’s a great Just William story where he sneaks into a waxworks exhibition and pretends to be one, and a young man comes along and points out all the defects in his utterly unrealistic anatomy. It was a bit like that.
If you weren't familiar then I could see it being hard.
A politician choses their speechwriters, works with them, and takes ultimate responsibility. The process just happens to be exactly the same as it for all other tasks they do. I would argue a demonstrated ability to choose good people and judge their work is so essential for high government office, the process involving speechwriters is a far better measure of their suitability for office.
Speechwriters, at least good ones, also don't write 'generic' texts. They will study their client's speech patterns and adjust word choices, rhythm, humor, and other patterns to fit the person's style. For offices as high as the US president, the writers tend not to change, sometimes over decades.
See the chart at the top of https://www.theatlantic.com/politics/archive/2015/01/the-lan... for word choices in the State of the Union. There are clear difference between and consistencies within each presidents' speeches.
For context: I'm mostly unaware of actions of US presidents (beyond the broad sweeps based on left vs right) and I've at most listened to Trump and Biden maybe a couple of times for a total of ~1min in the last few years - just doing this exercise has probably at least doubled my exposure.
I found the text and voice mostly impossible unless the content clued me in. Video was a little easier because I know to look for teeth, video with sound was fairly easy.
Is there a generic trick for recognising faked voices without really knowing the original (similar to looking for teeth on videos)?
Maybe we are missing some tricks to pick out the fakes.
For the majority of folks here who picked out the fakes easily, do you feel if you can do the same if the videos were of Putin speaking in Russian (with subtitles), or speaking in English with a very thick accent?
The fakes have completely clean audio with similar artificial noise (real ones have crowd, or other backround noise), the fakes have no microphone or environmental artifacts (echo/reverb, plosives, others).
What made them even easier to recognize is that accents were all wrong. They were exactly as another person said, caricatures of the original voices. I actually found that they were so obvious that most could be identified before the second or two that they make you wait to click the submit button.
If it were a foreign language, I think the success rate would be lower, but the unnatural lack of noise gives them away.
From the presented samples I'd wager that one goal of the study might be to quantify the influence of audio-only vs video-only vs text, hence text-only, video-only, audio-only, and video with subtitles.
While text limits the decision making process to preconceived notions about the person in question and prior background knowledge, video and audio may present clear cues from human perception alone.
Update:
Come to think of it, IRL text deep fakes don't even make sense. You can make up whatever quotes you like, you don't need GPT3 to do it for you.
Exactly. John Oliver pointed out a few weeks ago that the Russian media was using stock footage of other nation's military actions (Finland?) for propaganda. No need for deep fakes. The lowest effort lie can still work.
Deep fake tech is the new photoshop. It's mostly used for good (art, memes, jokes with friends, Disney films).
You can imagine a future where deep fakes are utterly indistinguishable from real life, but I think that people will learn to recalibrate themselves against the new technologies. It may even cause us to think more critically: people frequently chime in, "how is this not photoshopped?" on social media. They'll call out videos of explosions and physics as "looking fake". They'll have the same response to purported deep faked videos.
All the same, I'm glad and appreciate that academic and policy folks are taking a close look at the technology. We need frameworks, ethics, and institutional familiarity.
They make sense for advertising and misinformation purposes.
You don't have to pay a firm to drum up interest for a brand/product, theoretically, you'd be able to generate convincing conversations about the brand between bots on Reddit or Twitter, and do it at scale. Instead of paid posts on social media, you cut out the middle man and give the perception of community support for a brand/product, despite it being artificial.
The last point goes for misinformation, as well. Want to start grassroots support for an idea/policy/person/group/place? Instead of paying teams of people to post on social media about it, you can automate the campaign and scale it with deep faked text.
I don't think deep fakes are there yet for either purpose, though, and it isn't a given that they'll be good enough for them, either.
At that point its just standard propaganda. People have been doing that since roman times.
Anyway, THANK YOU site developers for properly handling resumption of progress and not abusing the history API. Greatly appreciated and unfortunately very rare for these sorts of sites.
They're still not particularly convincing, especially the audio ones sound very fake.
But I think these sorts of tests are flawed. The fact that it's a test primes you to look for these details in a way you might not if these clips were shared on Twitter/Reddit/etc. The deepfakes are still pretty obviously fake, so I don't think I'd fall for one "in the wild". But I also see this style of test used for things like "can you hear the difference between these 2 audio compression formats" or "does this virtual guitar amp sound just like the real thing", and that's just not how you consume real media. For example I'd probably never know Meshuggah switched to entirely digital guitar amps years ago if it weren't for them saying it in interviews.
When deepfakes one day do get as close to the real thing as digital guitar amps have gotten to real amps, I doubt I'd notice a random r/worldnews post is faked, even if I could still reliably pass an A/B test.
The reverse side of it is if an overwhelming amount of deep fakes get called out, people will stop trusting the real ones (we have already started stepping into that reality)
Or will they?
https://www.youtube.com/c/SassyJustice/featured
Better would be to let them see 10 videos all at once, half of which are fake, and ask them to divide into a fake set of 5 and a real set of 5, after looking at all of them as many times as they like. Asking "fake or real" when there is no basis to tell whether flaws should be taken as indicating "fake" or just attributed to compression seems meaningless.
Or tell people what aspects of fakeness they're trying to assess - eg, forget about video artifacts, just pay attention to the audio.
Using clips of Trump and Biden is also a bad idea. They ask you to say if you've seen one before, but aren't many people going to have seen one, but not clearly remember that, and then be influenced to think it's real by sub-conscious recognition?
Why not present pairs of videos of the same non-famous person, one fake one real, both presented with the same amount of compression, and ask one to chose which is the real one? Using many different people, of course - why would you introduce doubt about the generality of your results by using only two people?
Of course, in practice people may be less able to recognize fakes when video quality is poor, which would be useful to know, but I think one would need to investigate that issue separately, not in combination with other reasons that fakes might or might not be recognizable.
Ha it keeps on giving, note the picture on wall at https://youtu.be/9WfZuNceFDM
>Barack Obama reads the Navy Seals Copypasta (Speech Synthesis) https://www.youtube.com/watch?v=-_MZI2YFWgI
Or here him reading Trumps inauguration speech https://www.youtube.com/watch?v=ChzEdz7aVVs
All based on https://ai.googleblog.com/2017/12/tacotron-2-generating-huma...