I thought we were finally getting away from that "subtly flickering fluorescent lights" vibe in public spaces that gives 30% of the population headaches. But I guess we're bringing it back. Another victory for AI!
Have a vague memory of some old HN discussion about how known fluctuations in light because of slightly varying electricity frequency have been used already to detect fake video and that databases exist with information about frequencies by location and time for this purpose?
Without having read into this deeper, it sounds like someone could take an original video which has this code embedded as small fluctuations in luminance over time and edit it or produce a new video, simply applying the same luminance changes to the edited areas/generated video, no? It seems for a system like this every pixel would need to be digitally signed by the producer for it to be non-repudiable.
The code could be cryptographically derived from the content of the video. For simplicy, imagine there are subtitles baked into the video and the code is cryptographically derived from those.
While it does not seem enough to guarantee authenticity, this scheme does seem like it would prevent creating a video from scratch pretending to be taken at a protected location without having express knowledge of the key or the flickering at that moment in time.
Definitely interesting for critical event and locations, but quite niche.
> “Each watermark carries a low-fidelity time-stamped version of the unmanipulated video under slightly different lighting. We call these code videos,”
If this is the only info that's encoded, then that might not be an entirely bad idea.
(Usually, the stego-ing of info can help identify, say, a dissident who made a video that was critical of a regime. There are already other ways, but defeating them is whack-a-mole, if universities are going to keep inventing more.)
> Each watermarked light source has a secret code that can be used to check for the corresponding watermark in the video and reveal any malicious editing.
If I have the dissident video, and a really big computer, can I identify the particular watermarked light sources that were present (and from there, know the location or owner)?
It's rare that I think an academic paper from a good school that is trending on HN is actively stupid, but this is that paper.
If you're even considering going to go to all the trouble of setting up these weird lights and specialized algorithms for some event you're hosting, just shoot your own video of the event and post it. Done.
"Viewers" aren't forensic experts. They aren't going to engage with this algorithm or do some complex exercise to verify the private key of the algorithm prior to running some app on the video, they are just going to watch it.
Opponents aren't going to have difficulty relighting. Relighting is a thing Hollywood does routinely, and it's only getting easier.
Posting your own key and own video does nothing to prove the veracity of your own video. You could still have shot anything you want, with whatever edits you want, and applied the lighting in software after the fact.
I'm sure it was fun to play with the lights in the lab, but this isn't solving a problem of significance well.
A more accessible thing that protects against fake videos, at least in the short term, is multiple cameras and a complicated background.
Maybe eventually we get a model that can take a video and "rotate" it, or generate a 3D scene that can be recorded at multiple angles. But maybe eventually we may get a model that can generate anything. For now, 4o can't maintain obvious consistency with so many details, and I imagine it's orders of magnitude harder to replicate spatial/lighting differences accurately enough to pass expert inspection.
If you want solid evidence that a video is real, ask for another angle. Meanwhile, anything that needs to be covered with a camera (security or witness) should have at least two.
Even if everything they say is true, that wouldn't prove a video is fake, at best it proves a video is real. If people will accept "our high-profile defendant in the segregated housing unit of a maximum security prison hung himself with a makeshift noose fashioned from blankets off a bedpost that isn't even as tall as he is while the guards were playing 3d space cadet pinball and the camera was broken and his cellmate was in solitairy", surely they will accept "our maintenance guy used regular lightbulbs from home depot instead of the super secure digital signature bulbs".
Or maybe "we installed the right bulbs but then we set the cameras to record in 240p MPEG with 1/5 keyframe per second because nobody in the office understands how digital video works".
Anyways I'm of the opinion that the ultimate end-state of deep fakes will be some sort of hybrid system where the AI creates 3d models and animates a scene for a traditional raytracing engine. It lets the AI do what its best at (faces, voices, movement) and eliminates most of the random inconsistencies. If that happens then faking these light patterns won't be difficult at all.
I applaud the idea to mark videos as real, but somehow I don't think it matters. People disagree on facts and reality and ignore contradictions or proof to the contrary. If fact-cheking is already used as a slur or dog whistle in some circles, then what can a reality-watermark accomplish?
If it’s not directly human-verifiable, people have to rely on 3rd party tools or official/media statements to verify content legitimacy. Such reliance requires trust in authorities and media, which have both been subject to systematic erosion as of late.
I don’t see the point of this technology. It might be useful for entities like Meta and Google, which could use it to warn of fake content. However, in practice that amounts to giving those entities more power over our perceptions and the realities we build upon them.
I'm finding that AI seems incapable of generating aperiodic monotile designs. I suspect this is because the shape is nowhere in any training data, and it doesn't pattern--so without patterns to train on, it produces obvious errors. It invents geometry that stands out like a sore thumb. I think it has potential to serve as protection against deepfakes. I made an online store around all this, but I haven't really advertised it because I'd like a little more confirmation before I run with it. Would love some feedback on the idea
So, if the technical end of this really works, whoever controls the venue and its records can control what videos (real or fake) are considered “real”, in the rare circumstances where the kind of forensics needed to verify it would be employed.
Unfortunately, in most of the cases where that would be useful, that's also a party pretty high on the list of “who are we concerned about manipulating video”, so...
I'm not sure that raising the bar for how expensive it is to fake a video is the right step to take. That'll just preserve, for a little while longer, the false sense of security that comes from seeing a video and assuming it's real. Meanwhile, the well funded bad actors will remain uninhibited (and aren't they the scariest ones?)
It looks like fun research, but I think we'd get a lot better bang for our buck persuring ways to attach annotations to a video like:
> I was there and I saw this happen
...such that you can find transitive trust paths from yourself to a verifier who annotated the video. That'll require a bit of trust hygiene that the common folk aren't prepared for, but I dont think there's any getting around preparing them for it.
The problem of authenticating footage has a disappointing but sufficient solution:
1.- Use filming devices that sign the footage with a key, and the device has some anti-tamper protections to prevent somebody from stealing the key.
2.- The thing above is useless for most consumers of the footage, which would only see it after three to four transcodings change the bytes beyond recognition. But in a few years everybody will assume most footage in the Internet is fake, and in those cases when people (say, law enforcement) want to know for sure if something really happened, they’ll have to go through the trouble of procuring and watching the original, authenticated file.
The alternatives to 1 and 2 are:
a) To engage in an arms race, like the one which is happening with captchas right now.
b) To roll back this type of AI.
b is not going to happen, even with a societal collapse and sweeping laws against GenAI, because the way GenAI works is widely known. Unless we want to roll back technology itself and stop producing the hardware, and culture so that people don’t know how to do any of this.
This stopped me from playing any movie that had this implemented via PS3 Media Server on my PS3. Most movies that are done by Sony Pictures will have this implemented. I ended up using my Xbox 360 instead because Microsoft would not pay to license the tech IIRC.
23 comments
[ 3.0 ms ] story [ 45.0 ms ] threadI don't think there's any possible solution that cannot also be faked in itself.
Plus, the code gives information about the frame it's embedded into, so you still have more work to do.
Definitely interesting for critical event and locations, but quite niche.
If this is the only info that's encoded, then that might not be an entirely bad idea.
(Usually, the stego-ing of info can help identify, say, a dissident who made a video that was critical of a regime. There are already other ways, but defeating them is whack-a-mole, if universities are going to keep inventing more.)
> Each watermarked light source has a secret code that can be used to check for the corresponding watermark in the video and reveal any malicious editing.
If I have the dissident video, and a really big computer, can I identify the particular watermarked light sources that were present (and from there, know the location or owner)?
If you're even considering going to go to all the trouble of setting up these weird lights and specialized algorithms for some event you're hosting, just shoot your own video of the event and post it. Done.
"Viewers" aren't forensic experts. They aren't going to engage with this algorithm or do some complex exercise to verify the private key of the algorithm prior to running some app on the video, they are just going to watch it.
Opponents aren't going to have difficulty relighting. Relighting is a thing Hollywood does routinely, and it's only getting easier.
Posting your own key and own video does nothing to prove the veracity of your own video. You could still have shot anything you want, with whatever edits you want, and applied the lighting in software after the fact.
I'm sure it was fun to play with the lights in the lab, but this isn't solving a problem of significance well.
Maybe eventually we get a model that can take a video and "rotate" it, or generate a 3D scene that can be recorded at multiple angles. But maybe eventually we may get a model that can generate anything. For now, 4o can't maintain obvious consistency with so many details, and I imagine it's orders of magnitude harder to replicate spatial/lighting differences accurately enough to pass expert inspection.
If you want solid evidence that a video is real, ask for another angle. Meanwhile, anything that needs to be covered with a camera (security or witness) should have at least two.
Or maybe "we installed the right bulbs but then we set the cameras to record in 240p MPEG with 1/5 keyframe per second because nobody in the office understands how digital video works".
Anyways I'm of the opinion that the ultimate end-state of deep fakes will be some sort of hybrid system where the AI creates 3d models and animates a scene for a traditional raytracing engine. It lets the AI do what its best at (faces, voices, movement) and eliminates most of the random inconsistencies. If that happens then faking these light patterns won't be difficult at all.
I don’t see the point of this technology. It might be useful for entities like Meta and Google, which could use it to warn of fake content. However, in practice that amounts to giving those entities more power over our perceptions and the realities we build upon them.
Unfortunately, in most of the cases where that would be useful, that's also a party pretty high on the list of “who are we concerned about manipulating video”, so...
https://www.youtube.com/watch?v=e0elNU0iOMY
https://en.wikipedia.org/wiki/Electrical_network_frequency_a...
It looks like fun research, but I think we'd get a lot better bang for our buck persuring ways to attach annotations to a video like:
> I was there and I saw this happen
...such that you can find transitive trust paths from yourself to a verifier who annotated the video. That'll require a bit of trust hygiene that the common folk aren't prepared for, but I dont think there's any getting around preparing them for it.
1.- Use filming devices that sign the footage with a key, and the device has some anti-tamper protections to prevent somebody from stealing the key.
2.- The thing above is useless for most consumers of the footage, which would only see it after three to four transcodings change the bytes beyond recognition. But in a few years everybody will assume most footage in the Internet is fake, and in those cases when people (say, law enforcement) want to know for sure if something really happened, they’ll have to go through the trouble of procuring and watching the original, authenticated file.
The alternatives to 1 and 2 are:
a) To engage in an arms race, like the one which is happening with captchas right now.
b) To roll back this type of AI.
b is not going to happen, even with a societal collapse and sweeping laws against GenAI, because the way GenAI works is widely known. Unless we want to roll back technology itself and stop producing the hardware, and culture so that people don’t know how to do any of this.
https://en.wikipedia.org/wiki/Cinavia
This stopped me from playing any movie that had this implemented via PS3 Media Server on my PS3. Most movies that are done by Sony Pictures will have this implemented. I ended up using my Xbox 360 instead because Microsoft would not pay to license the tech IIRC.