Me as well, I thought to my self 'that dude has some resemblance to Tom Cruise' for a second and the next moment the similarity had dissapeared and I fully dismissed my initial reaction since he afterwards had no real resemblance.. Dismissed it and thought my mind had played a trick on me because of sleepiness or something. Only after glancing the video comments (after first full view) did I understand what was going on.
Same. I came across this clip a few days ago without the context.
It's a little embarrassing frankly. I kept thinking that he's such a good impersonator that he's able to manipulate his facial muscles in just such a way as to embody Tom Cruise.
Hypothetically, I meant, if someone were to style their hair as Cruise and did an entire video impersonating his speech - it would be convincing it was him
Totally, but that's exact same for our brain fooling us about the motion itself. And in motions case our brain throws in interpolated middle frames, not so in the face morph.
Does anyone know what models / software this person is using? I assume it's something off the shelf, because there are an abundance of other videos in this space produced by different people:
I've been doing ML for audio (Tacotron, WaveRNN, etc.), but haven't stepped into the video world. I also work in film and want to apply this technique to my work.
How do they keep the frames from blurring or not needing to be rekeyed? The motion tracking is spot-on.
Would someone operating in this space be open to meeting over Hangouts or Skype to discuss the tech? I'd be happy to pay someone for a short survey of the space.
This was in the news for being the first(?) public GitHub repo that you needed an account to access.
I believe to try and stem the tide of deepfake revenge porn or celebrity porn.
It isn't too hard to image a future where actors no longer sell their skills but rather just their name and likeness. And – similar to photoshop or autotune — expressions, dialog, posture and everything else can be edited as needed.
Assuming you mean they build up a name and then license their image ... I'm not sure you've met an actor before. This goes against every impulse of a great actor. They're completely obsessed with controlling their image and practicing their craft.
>They're completely obsessed with controlling their image and practicing their craft.
I'm sure textile workers said the same thing before their skill and craft was replaced by a machine. Every artist and artisan thinks they're irreplaceable. Currently, many programmers think theirs is the only industry that can't be automated because there's some magical golden juice in their minds that produces some creative and intellectual quality no machine will ever be able to replicate. Like textile workers of old, their time will come too.
As much as actors may want their work to be art, their work is also product. When it becomes possible to manufacture an acceptable equivalent product through automation, then artists of all stripes will simply have to adapt to the new terms of the market like everyone else, or find other work. No one is going to care about the effort an actor puts into their performance when it becomes feasible to pay a few dollars to stream Marlon Brando into the original Star Wars.
Elite actors of the world are extremely rare, talented individuals. They compete in one of the most cut-throat fields I can think of. How is that analogous to a textile worker?
As long as there are humans, then actors, musicians, dancers, etc will be fine. People are fascinated with artists because they demonstrate a spectacle of skill and a genius which we don't have.
>Elite actors of the world are extremely rare, talented individuals. They compete in one of the most cut-throat fields I can think of. How is that analogous to a textile worker?
The textile workers replaced by automation were also once extremely rare, talented individuals. Rugs were once prized works of art, infused with culture and artisanal techniques passed down through generations. Now you can buy them at Wal-Mart. Unfortunately, the post-industrial market doesn't optimize for skill, it optimizes against it.
>People are fascinated with artists because they demonstrate a spectacle of skill and a genius which we don't have.
Most people aren't fascinated with skill and genius, they're fascinated with celebrity. Art isn't made to be appreciated nowadays, it's mass produced to be consumed like fast food, with little regard beyond the immediate satisfaction it provides, and the familiar comfort of mediocrity.
Sure, there will always be a place for human artists, dancers, musicians, etc, just as there is still a place for people who can weave rugs. But I do believe that most mainstream art will be automated as soon as the result is acceptable to the mainstream, and that includes automating the persona of the artist as well.
Artisanal textile workers, categorically, were not as talented as elite actors because the size of their competitive pool and the barriers to creating work were minuscule compared to that of Hollywood actors - who emigrate from all over the globe and assimilate to US culture to use their talent.
While a lot of cultural products (not art) like movies and shows are created like fast food, people are drawn to them because of the unique talent, charisma, looks, presence etc of the actors that participate. I don't see Avengers: End Game breaking box office records if it didn't have those rarefied individuals.
If you go back to my initial post, I pondered whether the original comment meant celebrity.
Or maybe imagine a future were actors only sell their skills, since it's decoupled from their face, race, gender, and other appearance related factors.
Interestingly in that movie, it was Robin Wright and Tom Cruise who were the very last actors to hold out on selling their likeness for entirely generated films.
I would guess the opposite: the surface appearance is the easy part; articulating human mannerisms in a way that's not just convincing, but compelling, will remain one of the very hardest things for computers to replicate.
Acting, as a craft, only ever had coincidental ties to physical appearance. What it really is is the talent of being human. I think what we'll see is a lot of pretty faces who aren't actually that good at acting exiting the business, while upcoming talent won't have its career limited by birth attributes. It should be very interesting.
This video is helped by some superficial similarity between Hader and Cruise, and the transition to and from Seth Rogen's face is slightly less smooth. Still, this is incredibly well done.
Same user also did the Jim Carrey in Shining deepfake which is as well done as this and at times really unsettling. From now on I can safely say that I can't trust any video footage I see. This is that well done. The algos will only get better and do a convincing blend in 4K in the future.
I've been thinking a lot about this one, and it gets a ton of help in the fact that Bill Hader is actually good at facial impressions, and that's what really sells it.
Once that deep fake technology algorithm gets in the hands of the masses (Adobe Premiere After Effects plugin, Instagram/Snapchat filter, Apple Facetime filter, etc) ... so many obvious directions it will take:
- dating / flirting apps
- job candidates being interviewed remotely via webcams will use software to make them look younger
- "facelift" every television personality (news anchors, etc) and every movie actor. Both for creative purposes (younger/older timeline aging), for vanity, and for extending careers.
- generate fake but convincing political ads to put words in the mouth of opponent that he/she never said
- a growing distrust and skepticism of all videos including archive footage because we never had SHA256 authenticated hash of the old videos widely disseminated before deepfakes became ultrarealistic. E.g. There will be a subset of tomorrows kids that will not be convinced JFK ever actually said, "landing a man on the moon". The WTC towers were never hit by airplanes. All that old footage will be dismissed as fake. Video evidence becomes useless. Yes, society will try to counteract it with software the detects deep fakes but some of the public still won't be convinced. (Same as evidence against anti-vaccine being rejected.)
> - growing distrust and skepticism of all videos including archive footage
That's an excellent, and very terrifying point. Governments are going to have a hay-day with misinformation campaigns to rewrite history (even more than they do now).
On the first day, God created The Deepfake Algorithm. On days 2-6, he waited patiently as the The Algorithm auto-generated history up to and including yesterday. Day 7 is today.
1. We don't have videos of The Gettysburg address. History existed before video.
2. There will never be another generation (in the near future) like boomers who didn't grow up tech savvy
3. Videos, like any form of documentation, are anecdotal evidence easily manipulated by what's "not in the frame"
4. The general problem you are describing comes from a faulty attribution of truth to video in our current day and age. The sooner we dismiss that the better
Yes, the transition will be awkward, but no need for hysteria.
I don't know if there's need for hysteria, but I think this is a much bigger change than you're making it out to be. It's a different league than artful editing and framing.
I can't tell if you both are joking or not but watch the whole video. The source video is of Bill Hader. Bill hader's face changes to Tom Cruise everytime he talks from Tom's perspective. And then once to Seth Rogan for that impersonation.
You’re correct. If you listen to what he’s actually saying it’s told from the perspective of Hader and it only shifts when he’s doing an impression of Cruise or Rogan.
As an aside, I feel really weird having completely convinced myself that the original video was deepfaked. It's kind of unnerving given the subject matter.
I honestly can't see much change in the face throughout the video.
It probably doesn't help that I am so horrible at celebrities that I couldn't picture Tom Cruise or Bill Hader in my head if I tried, nor do I know what they're famous for. I just know that Tom Cruise sounds familiar and everyone seems to act like Bill Hader is significant so I go along with it.
I guess there's a some change in the face when he switches "voices". It's not nearly as exciting as the one of Obama saying stuff he's never said before though.
If you know Bill Hader's face, the change to Cruise is pretty obvious. The Cruise resemblance is subtle but strong.
Part of the point with fakes like this is it's not just gluing a head on someone else's body, like a bad photoshop. It's actually changing the facial features of the target to resemble someone else.
Same creator as "Bill Hader impersonates Arnold Schwarzenegger", which I think is easier to see the shift in since there's less resemblance and higher quality: https://www.youtube.com/watch?v=bPhUhypV27w
That's an interesting idea; how would you go about suggesting we do that? Have some kind of hardware-signing thing that we put into video cameras, and sign every keyframe with it before its written to the SD card?
Eh, photoshop and PDF editors exist and we can still use documents and photos as evidence.
But, there will definitely be a new cohort of expert witnesses with software tools (functional or just appearing so) to testify about the veracity or video.
Only on LSD have I seen faces change that seamlessly. There were a couple of spots that I noticed the transition, but that was like the 3rd viewing. I hope I never piss off someone with these abilities to make me say/do things I'd never imagine. The damage that can be done is mind boggling when used for that purpose.
Photos do not lie, it is said. But frequently they tell partial
truths. They tear moments out of time's flowing river and separate
them from what has come before and what will follow. They can
misrepresent as well as represent.
- JIM HOAGLAND
72 comments
[ 5.0 ms ] story [ 141 ms ] threadIt fooled me on the first view, I just didn't even see it, it was seamless.
It's a little embarrassing frankly. I kept thinking that he's such a good impersonator that he's able to manipulate his facial muscles in just such a way as to embody Tom Cruise.
My defenses are now up a bit higher.
https://www.cnet.com/news/parks-and-recreation-ron-swanson-d...
I've been doing ML for audio (Tacotron, WaveRNN, etc.), but haven't stepped into the video world. I also work in film and want to apply this technique to my work.
How do they keep the frames from blurring or not needing to be rekeyed? The motion tracking is spot-on.
Would someone operating in this space be open to meeting over Hangouts or Skype to discuss the tech? I'd be happy to pay someone for a short survey of the space.
This was in the news for being the first(?) public GitHub repo that you needed an account to access. I believe to try and stem the tide of deepfake revenge porn or celebrity porn.
Peter Jackson's Bolg and Azog could have been much more convincing, just 7 years on.
Assuming you mean they build up a name and then license their image ... I'm not sure you've met an actor before. This goes against every impulse of a great actor. They're completely obsessed with controlling their image and practicing their craft.
I think you may have meant celebrities.
I'm sure textile workers said the same thing before their skill and craft was replaced by a machine. Every artist and artisan thinks they're irreplaceable. Currently, many programmers think theirs is the only industry that can't be automated because there's some magical golden juice in their minds that produces some creative and intellectual quality no machine will ever be able to replicate. Like textile workers of old, their time will come too.
As much as actors may want their work to be art, their work is also product. When it becomes possible to manufacture an acceptable equivalent product through automation, then artists of all stripes will simply have to adapt to the new terms of the market like everyone else, or find other work. No one is going to care about the effort an actor puts into their performance when it becomes feasible to pay a few dollars to stream Marlon Brando into the original Star Wars.
As long as there are humans, then actors, musicians, dancers, etc will be fine. People are fascinated with artists because they demonstrate a spectacle of skill and a genius which we don't have.
The textile workers replaced by automation were also once extremely rare, talented individuals. Rugs were once prized works of art, infused with culture and artisanal techniques passed down through generations. Now you can buy them at Wal-Mart. Unfortunately, the post-industrial market doesn't optimize for skill, it optimizes against it.
>People are fascinated with artists because they demonstrate a spectacle of skill and a genius which we don't have.
Most people aren't fascinated with skill and genius, they're fascinated with celebrity. Art isn't made to be appreciated nowadays, it's mass produced to be consumed like fast food, with little regard beyond the immediate satisfaction it provides, and the familiar comfort of mediocrity.
Sure, there will always be a place for human artists, dancers, musicians, etc, just as there is still a place for people who can weave rugs. But I do believe that most mainstream art will be automated as soon as the result is acceptable to the mainstream, and that includes automating the persona of the artist as well.
While a lot of cultural products (not art) like movies and shows are created like fast food, people are drawn to them because of the unique talent, charisma, looks, presence etc of the actors that participate. I don't see Avengers: End Game breaking box office records if it didn't have those rarefied individuals.
If you go back to my initial post, I pondered whether the original comment meant celebrity.
Physical features will be cosmetic and plastic, and they will be engineered to match the desired setting.
https://www.youtube.com/watch?v=Cy2WLpGxGVI
Acting, as a craft, only ever had coincidental ties to physical appearance. What it really is is the talent of being human. I think what we'll see is a lot of pretty faces who aren't actually that good at acting exiting the business, while upcoming talent won't have its career limited by birth attributes. It should be very interesting.
https://youtu.be/aUphMqs1vFw [Full House of Mustaches]
- dating / flirting apps
- job candidates being interviewed remotely via webcams will use software to make them look younger
- "facelift" every television personality (news anchors, etc) and every movie actor. Both for creative purposes (younger/older timeline aging), for vanity, and for extending careers.
- generate fake but convincing political ads to put words in the mouth of opponent that he/she never said
- a growing distrust and skepticism of all videos including archive footage because we never had SHA256 authenticated hash of the old videos widely disseminated before deepfakes became ultrarealistic. E.g. There will be a subset of tomorrows kids that will not be convinced JFK ever actually said, "landing a man on the moon". The WTC towers were never hit by airplanes. All that old footage will be dismissed as fake. Video evidence becomes useless. Yes, society will try to counteract it with software the detects deep fakes but some of the public still won't be convinced. (Same as evidence against anti-vaccine being rejected.)
That's an excellent, and very terrifying point. Governments are going to have a hay-day with misinformation campaigns to rewrite history (even more than they do now).
1. We don't have videos of The Gettysburg address. History existed before video.
2. There will never be another generation (in the near future) like boomers who didn't grow up tech savvy
3. Videos, like any form of documentation, are anecdotal evidence easily manipulated by what's "not in the frame"
4. The general problem you are describing comes from a faulty attribution of truth to video in our current day and age. The sooner we dismiss that the better
Yes, the transition will be awkward, but no need for hysteria.
You're living in a bubble.
(With making-of.)
As an aside, I feel really weird having completely convinced myself that the original video was deepfaked. It's kind of unnerving given the subject matter.
It probably doesn't help that I am so horrible at celebrities that I couldn't picture Tom Cruise or Bill Hader in my head if I tried, nor do I know what they're famous for. I just know that Tom Cruise sounds familiar and everyone seems to act like Bill Hader is significant so I go along with it.
I guess there's a some change in the face when he switches "voices". It's not nearly as exciting as the one of Obama saying stuff he's never said before though.
Part of the point with fakes like this is it's not just gluing a head on someone else's body, like a bad photoshop. It's actually changing the facial features of the target to resemble someone else.
Yeah, you can see occasional artifacts and glitches with current deepfakes, but surely the software will only get better.
We should add integrity checks and digital signatures to videos.
But, there will definitely be a new cohort of expert witnesses with software tools (functional or just appearing so) to testify about the veracity or video.