While I share your reservations, I think we should either give it a whirl or wait for someone else to try adversarial processing before we dismiss it out of hand.
My guess is though that if you had a training set of processed vs unprocessed pairs that you could train an “unmister” if you wanted to.
Not much point in giving it a whirl when the resulting images are clearly not something you'd put in your site to show off your work. The problem with this (and glaze, etc) is that it does what it claims by ruining the image it's used on.
Unfortunately it's the natural progression of the web, given the unethical ways people are training their models. This is going to happen in more ways than just images.
I think the thumbnails of the misted look awful, though that might be because I can see the original to compare. There seems to be weird semi-regular patterns around the images that I think I'd notice even without the comparison, though.
If it's anything like Glaze, it's a complete meme. That took a lot of compute to glaze an image (with very visible artifacts) and was bypassed with some trival image processing on a shitty laptop.
This is incredibly similar to a paper one of my professors was working on a year or two ago: [0]
> Neural Trojans embedded in pre-trained neural networks are a harmful attack against the DNN model supply chain. They generate false outputs when certain stealthy triggers appear in the inputs. While data-poisoning attacks have been well studied in the literature, code-poisoning and model-poisoning backdoors only start to attract attention until recently.
While I agree with other commenters here that it's hard to imagine it being widely implemented at the moment, I think it's only the beginning of this type of tech. In a few years, maybe this type of data poisoning could be as ubiquitous as SSL and as subtle as webp compression artifacting.
These tools do not work in the real world for several reasons: first, people who train LoRA or models curate their datasets, removing adversarial noise is trivial: https://github.com/lllyasviel/AdverseCleaner ; second, if they are trying to defend themselves against training AI models, then it will probably do almost nothing at all, models trained from scratch will learn the distribution, even one where the samples have adversarial noise (which can only attack a frozen model, like a VAE), SDXL has a new VAE, so the VAE will just be more robust if there is adversarial noise on the images, because it will learn to ignore it, many other models do not have a VAE to begin with.
Also, resizing the image (as is almost always done when training the model) will probably destroy most if not all of the adversarial noise.
It’s hilarious that we all know DRM for video is doomed because anything someone can see can be copied.
And yet people think that somehow visual art is different; that there must be some technical solution that will keep the wrong people/computers from being able to see it, while letting the right ones in.
It is not now, and it never has been, and it never will be possible to have widespread distribution and absolute control over copies.
Even if you went to such extreme lengths as encrypting it such that decryption happened on monitors, a camera with a high enough resolution could still just make an (imperfect) copy.
Scanners and printers that don't scan/print currency have been around for decades. It doesn't seem that much of a stretch to have a "do not duplicate" watermark embedded in audio/video streams that recording devices respect.
I could imagine (say) the latest and greatest video encoder consortium requiring that as a condition of licence.
This might seem overly pessimistic, but I feel like the relationship between the customer and the computing device is steadily tilting towards "consumer / user" vs "customer / owner"
The difference there is that printed money has to be very high quality to be worth copying, so you can enforce it on the few companies which make good enough printers.
A camera on the other hand is very cheap to make. You could maybe enforce all smartphones to obey your watermark, but what about a raspberry pi?
Because if the Raspberry Pi camera can not take photographs that support the current iPhone coolness (live photo, or depth, or on-viewing selectable dynamic range, or I don't know what else) because the latest codec is not licensed, then the camera will not sell. So the quality Raspberry Pi camera makers don't even sell a model that doesn't respect the copy protection.
Only need to deprecate the ability to create such an image on the camera, not the ability to read one.
Example license phrasing:
> To license Foobar, the licensee will refrain from manufacturing or selling devices which could be used to circumvent SuperFoo Security features, even if such functionality is not part of a device supporting Foobar itself.
Or, for instance, it could require SuperFoo watermark detection to run continuously, regardless of whether the encoder output is SuperFoo Codec or something like MJPG.
I think it is naive to assume that such DRM will always be crackable. I was at a recent conference on AI vision, and one big (to me) new feature that camera sensor vendors like Sony are advertising are ISP-like features directly on sensor (motion detection etc). At some point, it seems entirely possible that you couldn't even bypass something like this at the sensor level.
Frankly, I see AI training on publicly available images to learn a style which it will then reproduce similar to the problem of game clones.
Imagine spending months/years making a game, coming up with the concept and gameplay, tuning the gameplay, … then after release cloners makes a nigh exact copy with slight changes in artwork/text and sell it for 10% of what you are selling - which they can since they put in only a fraction of the effort - and there is nothing you can do about it since gameplay isn’t copyrightable.
Same thing here IMHO. Not an AI specific problem but AI takes “no effort” when replicating an artist’s style vs a human who at least has to practice and learn the style.
AI art seems great if you want to generate an 80% there stream of generic waifus, but turns out nobody wants to see that. It's like buying a coffee mug from the store and wanting praise as if you fired it yourself.
46 comments
[ 2.7 ms ] story [ 81.4 ms ] threadcan it work for closed models, like midjourney?
My guess is though that if you had a training set of processed vs unprocessed pairs that you could train an “unmister” if you wanted to.
I look forward to the masterclass in ethical justification acrobatics that would accompany such an effort
Previous adversarial methods I've beat with simple noise/blur. This one is fairly resistant to basic filters.
Their example Van Gogh images look okay as thumbnails but are really different from the originals when viewed larger.
Overall a neat project though.
And you'd be spending that compute time for what will be a tiny, tiny,portion of that
> Neural Trojans embedded in pre-trained neural networks are a harmful attack against the DNN model supply chain. They generate false outputs when certain stealthy triggers appear in the inputs. While data-poisoning attacks have been well studied in the literature, code-poisoning and model-poisoning backdoors only start to attract attention until recently.
While I agree with other commenters here that it's hard to imagine it being widely implemented at the moment, I think it's only the beginning of this type of tech. In a few years, maybe this type of data poisoning could be as ubiquitous as SSL and as subtle as webp compression artifacting.
[0]: https://dl.acm.org/doi/abs/10.1145/3548606.3560678
It won't be hard to train models to recognize and compensate for this and other permutations of it. Meanwhile you've uglified your art piece.
Also, resizing the image (as is almost always done when training the model) will probably destroy most if not all of the adversarial noise.
And yet people think that somehow visual art is different; that there must be some technical solution that will keep the wrong people/computers from being able to see it, while letting the right ones in.
It is not now, and it never has been, and it never will be possible to have widespread distribution and absolute control over copies.
I sure hope this is true, but this strikes me as a bold statement given the ongoing war on general-purpose computing.
I could imagine (say) the latest and greatest video encoder consortium requiring that as a condition of licence.
This might seem overly pessimistic, but I feel like the relationship between the customer and the computing device is steadily tilting towards "consumer / user" vs "customer / owner"
https://en.wikipedia.org/wiki/EURion_constellation
A camera on the other hand is very cheap to make. You could maybe enforce all smartphones to obey your watermark, but what about a raspberry pi?
Example license phrasing:
Or, for instance, it could require SuperFoo watermark detection to run continuously, regardless of whether the encoder output is SuperFoo Codec or something like MJPG.
I think it is naive to assume that such DRM will always be crackable. I was at a recent conference on AI vision, and one big (to me) new feature that camera sensor vendors like Sony are advertising are ISP-like features directly on sensor (motion detection etc). At some point, it seems entirely possible that you couldn't even bypass something like this at the sensor level.
There may be a mild deterrence to casual users, but if the photons are reaching a human’s eyes, there are ways to capture them.
This is basically already how it's done fyi.
https://en.m.wikipedia.org/wiki/High-bandwidth_Digital_Conte...
I totally believe that the folks who don’t care about artist wishes will find a way around this, but it’s a game we’ll have to let play out.
This wouldn't remove the artists work from the database. (Unless you consider the adversarial noise a meaningful part of the art...)
Frankly, I see AI training on publicly available images to learn a style which it will then reproduce similar to the problem of game clones.
Imagine spending months/years making a game, coming up with the concept and gameplay, tuning the gameplay, … then after release cloners makes a nigh exact copy with slight changes in artwork/text and sell it for 10% of what you are selling - which they can since they put in only a fraction of the effort - and there is nothing you can do about it since gameplay isn’t copyrightable.
Same thing here IMHO. Not an AI specific problem but AI takes “no effort” when replicating an artist’s style vs a human who at least has to practice and learn the style.