Generally, the VAE is mapping from a small latent space to a large image space. This means that there must be a large number of images for which no reverse mapping exists.
It should be possible to identify images that have not been generate by the VAE since they are not part of the set images that the VAE can generate. The other way round is a bit more difficult as there may be images that can be mapped to the latent space and back without loss but have been generated in another way
This logic has a key flaw: just the fact that the size of the space is different doesnt mean that every representable thing in the larger space is a thing we care about. E.g. a person with three hands may not have a representation in the smaller space, but we would never care about that. The actual question is: what is the difference in the amount of information encoded in a large image vs the small latent space and compare that to the difference in information between a large image and a small image. If those two differences are close enough together, being able to determine a legitimate difference between SD generated vs not becomes near impossible.
The logic is still the same. If the VAE is trained so that it is biased toward human preference, then the probability of false positives in real world images would increase.
Yes. Trying to sell an "AI detector" is a fool's errand since this is how adversarial networks (the hot model from two hype cycles ago) are trained. The only use of the "AI detector " is to tune the model so that the detector is uniformly random.
This is true for explicitly adversarial actors. I can imagine a serious-business misinformation group investing the money required to train up a generator designed to defeat automatic detection.
For everyone else though, the people making common tools everyone uses explicitly want their images to be easy to automatically ID as fake. And, the users largely prefer it that way too.
As a user, I don't really care one way or the other. I'm not trying to trick anyone. I also don't care if they can readily tell if my image was AI generated. I don't post much media though. If I did, I'd probably use it for a game. And if it's a game, does it matter if I touched it up in Photoshop vs StableDiffusion? There's going to be some processing either way to get the art asset I need.
The toolmakers might have a mild incentive to tag images generated with their tool to prevent too easily creating these 'fakes' or just for analytics purposes so they can see how their output is spreading.
But regardless, it's a fools errand like the other poster said. Anyone that is serious about tricking the mass media will strip out the watermark or use a different tool.
Telling how "AI-like" an image is is done by seeing how well it completes a round-trip through the VAE.
It will either match a known VAE well, or be an image that's altered enough (jpeg noise, photoshopping in post, etc) to not match the output of a VAE.
And meanwhile the "loss system" for training AI idea is still hobbled by how the image generating AI systems are all using VAEs. At best, you've swapped one VAE for another. Just another new VAE to add to the list of VAEs to check.
24 comments
[ 5.4 ms ] story [ 68.6 ms ] threadIt should be possible to identify images that have not been generate by the VAE since they are not part of the set images that the VAE can generate. The other way round is a bit more difficult as there may be images that can be mapped to the latent space and back without loss but have been generated in another way
-> there may be false positives.
Of course, you will need to remove exifs or other metadata, but this sounds like the kind of domain that NNs are good at.
For everyone else though, the people making common tools everyone uses explicitly want their images to be easy to automatically ID as fake. And, the users largely prefer it that way too.
The toolmakers might have a mild incentive to tag images generated with their tool to prevent too easily creating these 'fakes' or just for analytics purposes so they can see how their output is spreading.
But regardless, it's a fools errand like the other poster said. Anyone that is serious about tricking the mass media will strip out the watermark or use a different tool.
It will either match a known VAE well, or be an image that's altered enough (jpeg noise, photoshopping in post, etc) to not match the output of a VAE.
And meanwhile the "loss system" for training AI idea is still hobbled by how the image generating AI systems are all using VAEs. At best, you've swapped one VAE for another. Just another new VAE to add to the list of VAEs to check.