I wonder if you can build a classifier to detect those attacks by nightshade. Either way it's clear that these supply chain attacks need to be taken more seriously. I wonder how come humans aren't that prone to this adversarial nosie compared to models
What makes you think we aren't as prone to this as the models are? There is a good reason folks rarely notice the many animals that are surrounding them on a walk in the woods.
Even worse, we are prone to being influenced by the decor or surroundings in ways that we know, but still partake in. Mood lighting would be an easy example. And we are very influenced by our other senses at the same time as purely visual. (Though, I'm always amused by how much the VR experience of racing a car hits, all without too much haptic feedback from the steering wheel.)
There was an attempt at poisoning training data with such invisible cloaking before and it turned out that this didn't really affect training quality (Glaze was the project name)
Iirc there was also some license violation with the cloaking code ironically, it was like a pillow fight
The ML community is stuffed with license violations. I have seen more than once instance of randomly copy-pasted code by sheer coincidence, recognizing it from older projects.
ML researcher culture is hard to describe... But its something like "Everything is a research prototype, so why does it matter?"
> There was an attempt at poisoning training data with such invisible cloaking before and it turned out that this didn't really affect training quality
Source? The authors claim this is not the case, and they have experiments showing it.
Which big players? There are some big players with an interest in facilitating the pervasion of AI, sure, but as far as I can tell there are are least as many big players who stand to lose from the widespread erosion of intellectual property rights that comes with it.
Publishers, record labels, movie studios, basically all of the businesses that have spent decades lobbying to strengthen the intellectual property laws that are now being undermined.
Freedom of speech in the US? I'm gonna laugh all the way into negative internet happy points that are gonna teach me a lesson never to say this again, unless this comment gets removed by the local censor :)
The first amendment is going to slap the taste out of their mouths if they try it. If you want to steal my data for your personal use, I am under no obligation to play nice with you.
It's illegal to booby-trap your house with the malicious intent to kill trespassers. With enough lawyers/lobbying, the same argument could work to criminalize the intent to break the product, even if it violates your license (separate offence).
Here in the real world, booby-trapping a house is entirely different than ensuring that your data set is fucked up when you're so lazy that you will just steal from everybody.
You will never get courts to conflate the two issues because they are fundamentally different. I have a constitutional right to my speech. You do not.
Calm down please, you seem to be misunderstanding my comment. I don't justify training without consent, I explain how the argument of the grandparent can be applied in reality. Yes, with enough legal backing they can sue you for the damages if they can prove the intent to poison their model. This intent doesn't have anything with freedom of your speech or the copyright. They might violate the license for your artwork and you can try suing them, but you are also not within your right to damage the rest of their work (which this thing attempts to do). Also the word "theft" can't be legally used in court in copyright infringement cases.
You just conflated homemade booby traps with stealing data, my guy. I am under zero obligation to ensure that my works are compatible with your models. They have the obligation to ask. I don't have the obligation to seek them out to give consent.
But I'll humor you. Let's assume that they sue me. Know what I'm gonna do? I'm going to sue them for the value of their entire corporation for stealing my work.
I don't understand what you're saying here. Why are you upset about creating new tools for current and future humans to be more capable at unleashing their creative visions?
All progress comes at labor from the past. If you didn't utilize past labor, you would not be talking to us now on the internet. You would be banging rocks against some cave wall instead.
If someone is inspired by art and they become an artist creating a similar style, is that stealing from you too? Is using a computer stealing from the engineers that created them? Is sleeping in a house stealing from the architects and builders?
If someone speaks do they need to pay royalties to the creators of the English language?
This technology is the beginning of creating a real life holodeck. Anything someone imagines they can creatively bring to life. How is it possible there are people greedy and self centered enough to not desire that for the future?
It is hard to follow the logic of your stance. I am genuinely curious about your modality of thinking if you would care to elaborate.
This whole "shut up about your intellectual property rights and your ability to make a living being a creator, you greedy self-centered idiot, because your livelihood is getting in the way of my vision of the future" is not convincing to me. Especially when your vision of the future is literally based on science fiction and may never actually come to pass. More likely gen AI will be like every technology, whose inventors and fans promise a post-scarcity world, but in reality it just either makes the rich already richer or swaps out one ruling class for another. We were promised the holodeck and the replicator, and what we got was Amazon drivers having to pee in bottles.
Does someone in a third world country who came up with the exact same idea as you years earlier have "intellectual property rights?" Is that entire conception not just construct of privilege, safeguarding only the well off?
Generative AI is already allowing people to unleash their creativity in miraculous ways, thought impossible previously. I feel your argument is unfortunately extremely dated already. I am also not seeing your connection between holodeck and replicators to Amazon drivers peeing in bottles? I wonder what coal miners have to do to make a living? There are all kinds of grotesque jobs and practices, the aim is to eliminate all of them in time with automated labor, and taking all the necessary supportive actions to make that possible / sustainable. Generative AI distributes creative control and power to everyone instead of those with money to afford it.
I have a right to the things I create. I have a right to modify them. I have a right to destroy them.
You do not. So if you come to steal from me, do not be surprised when I poison the living shit out the well you're attempting to drink from.
The greed here is not from the content creators, but from the wannabe AI overlords who believe they have the right to profit off of other people's work without their knowledge or consent.
You want data to train your AI? Either create it, buy it elsewhere, or pay me according to MY terms. But you won't get it from me for free and I will purposefully make your life hell if you attempt to do so. I don't give a damn about whatever pie in the sky nonsense you can come up with about technology and advancements. Your desire to get rich in the future is not more important than my need to eat today.
Yes, in the same way that the junkie dies from an OD because he steals random unidentified drugs in search of a fix, only in this case, they never had a right to any of my work.
Unfortunately, the technology is here and your skills are irrelevant. Coupling beautiful creative energy with money and greed was a mistake that you have had many opportunities in life to course correct from. Now you have to adapt. Sorry.
Looks like that type of art gets to be a passion project for people again instead of a way to make a living. Creative, passionate people will still create regardless of the machines. The machines will do all the soulless breadwinning artwork. I am and will continue to celebrate this change where raw and pure creative expression is accessible to more people.
If you don't want to adapt, I am sure your financial needs will necessitate getting a new skill you can still monetize, hopefully one that doesn't involve flipping burgers because of a hard-nosed stance (nothing wrong with burger flipping, it is just those jobs will be done by robots soon too).
The technology is trash and the "work" produced by these things is pathetically bad. And it's always more fun to make gobs of money by becoming a copyright troll to AI companies than it is to do it the old fashioned way. Wanna know where my proof that they've been stealing my work is gonna be? When they come at me for fucking up their models.
Nah, they'll use the easiest/cheapest/fastest way to prevent this, and, in this case, I'm pretty sure that is not litigation, but technical. I'm pretty sure they will be able to filter this stuff out or minimize its impact on training.
This isn't the first such poisoning method published is that they are announced, and then quickly its shown that (1) they aren’t really visually acceptable for most real art when you look at images at a bigger size than what is included in the announcement paper, resulting in noticeable quality degradation and/or (2) they can be easily countered, usually, IIRC, both.
I think it seems very likely that that this will always be the case: if it is visually neutral enough to be acceptable for use, there will be a way to train systems to generate art that visually resembles the original based on the output.
It's the way forward though. AI won't stop like this, psychotherapy will be next in the coming years (look at Pi, it has a therapy mode now even). Just accept that AI will affect all aspects of life in 10 years
Toward the future and whatever it holds, is the usual meaning of that phrase.
I think their point might be that we need to figure out how we live in a future with AI in it. Trying to deny AI with tools like this is not going to work.
Towards one man game devs being able to execute their vision, to middle class families being able to afford custom bespoke art, artists being able to greatly accelerate their workflows, etc.
You know, technological progress. We don't stop it when it affects accountants, delivery drivers, programmers, etc. Art should be no different.
Yeah, well, fuck Photoshop too. And video editors.
Jerks are going to find dumb, lazy, malicious ways to abuse good tools, like they always do, though diffusion does have an conspicuously low effort barrier.
Artists are still able to produce art with or without photoshop. Without their LLMs and the stolen art within, the best an AI "artist" can do is to scrawl "Tracer Overwatch big boobs not ugly trending on artstation" into a page.
> Artists are still able to produce art with or without photoshop
And that's was some actual AI artists do too. Its massively turbocharging stuff thats they can do in an illustrator or photo editor, but just takes forever. The tooling around diffusion is a deep rabbit hole, and requires some skill to use.
There's an important distinction between that kind of artist and the "AI users" flooding the internet. They are more akin to dank meme makers using Photoshop.
Not sure if that's happening at scale. AI-photobashing were and are attempted, but it seems AI always tries to undo changes so to bring deviations down to optically correct median form(as AI understands it), including all artist intentions. This applies to overall composition of the art as well.
Being forced into an "average" grade sounds okay, but rather, I think, what is happening is that a new baseline for minimum meaningful art was created, and at the same time, new definition for value of art is being established as errors from that baseline. And I'm not sure those are good things.
AlphaGo was trained on millions of professional Go matches. AlphaGo Zero trained purely by playing itself, with no exposure whatsoever to human play; it ended up being easier to train and much stronger.
I'm not saying that we will soon have a state-of-the-art image generation model that relies on self training, supervised learning or adversarial learning rather than a scraped data set, but I think it's a highly likely outcome within the next few years. Anyone making the argument that AI art is just copying might be in for a rude awakening. If that comes to pass, will you eagerly embrace AI art?
Thats not really equivalent, as image diffusion models are training on "real world" styles and topics, not a constrained game. Additionally, AlphaGo outputs are quantifiably "flawless," whereas GenAI models have all sorts of artifacts and distortion in their output that can't possibly be reliably filtered, even with an army of humans to do it.
My experience with ESRGAN is that it really does not like to train on its own output. It amplifies all sorts of wierd artifacts you can barely see I'd you go back and squint at the finetuning dataset. And there was a paper suggesting newer generative models are similar.
Of course I won't. Art at some level is meant to be communication between the artist and the observer/reader/listener. What communication is there in AI art besides "The algorithm did it"?
That's like saying "fuck computers" because they automate typists, elevator operators, and Rolodex company salespeople.
To quote an artist that I know, "artists already steal. They're the masters at it."
Stealing inspiration and style is how artists learn. It just takes time. They're angry that the time investment is going to zero. And also that their examples are what are being used, despite the fact that other human artists are doing that already.
I'm not sure we should defend any difficulty or complexity. Even that of our own careers.
We'd still be churning our own butter if we were afraid of automation. And what time would that leave us with?
There might be an ethical space in which AI imagery can exist, but consent is a direly needed component.
The typical argument against this is that human artists and ML models learn the same way, but the problem is that there's a fundamental difference: humans produce art by abstracting the concepts (think directional lighting, the kinematics of skeletal creatures, color theory, or physics of cloth) from the pieces they've studied and combining that with what they've learned from observing the world around them and by producing original art. This is why a human artist can produce competent art after studying only a handful of art pieces and gain most of their skill through practice.
By contrast, ML models do not learn by producing and have no way to directly observe the world. They don't abstract concepts, they "learn" by memorizing huge numbers of pre-existing images, and they produce images by patching together tiny bits and pieces of of those memorized images.
Where the human artist can function on little to no input, an ML model cannot, and so it's only fair that human artists have the choice to not have their works included in training datasets because if there is no feedstock of art there is no AI generated imagery.
The middle class are starting to experience what the working class have spent the last two centuries enduring. You're a skilled artisan with years of training and experience? Tough luck, we've got a machine that can do your job better, faster or at least cheaper.
I have a clog you can throw into a loom if it'll make you feel better, but there's no stopping AI art.
Sounds interesting, but without more details I'm not sure whether it would be so hard to detect and even correct for. Might be defeated even by some pixel averaging.
These are temporary measures though. By most respects, training against data with this type of poisoning is going to improve the overall performance of models because they'll necessarily have to learn to disregard this sort of invisible data (which necessarily it's far a field from human perception).
With a mere moment's thought, one thinks of the obvious way to defeat this:
Simply rescale all images in a preprocessing step.
Since this defeat is so easy, and so obvious, either the authors are daft, or they are brilliant and have thought of a way to prevent it.
So, not wanting to waste much time on a paper from daft people, I did a quick skim of the paper. I don't see any mention of this defeat. No prevention of it, no assessment of it, not even any evidence of awareness of it.
Did I miss something?
I'd hate to see technically naive artist types (a mere subset of artist types, I acknowledge) being hoodwinked by such snake oil and paying money for such "tools."
Ever so slightly degraded perhaps in some cases but "blurry" is overstating it. And worth the trivial cost if such dumb-as-bricks techniques take hold.
One could also submit each image to an image-to-image model with a blank prompt and the right settings to essentially ask for almost a copy, and come out with washed images. Could even do a sharpening (or upscale and sharpening for that matter) at the same time. It has a small (tiny really) GPU cost per image but it's a one-time cost.
You could use it to check if the image is poisoned. Original image is a "cat", downscaled/blurred/posterized or whatever is a "dog", then something is strange and better skip it.
I think “whatever” in your sentence is actually “have an infallibe human who knows all existent artistic styles tag it accurately with subject and style descriptions and then fine tune or train a model with it and then see if the model prompted with those descriptions produces images that match those descriptions.”
So I think simply rescaling would be a lot easier.
As with any DRM technology, the most it can hope to accomplish ultimately is to put more barriers and risks in the way. It won't stop the unwanted behavior completely, but provided it does what it says it shouldn't have zero impact either.
That sounds like a lot of work for something that can be worked around trivially. Add a small amount of random noise to the images and you’re done. Most people are probably doing this already for data augmentation purposes.
I always had this idea in my head about feeding AI garbage it cannot regurgitate into anything useful but not a clue on how to create said garbage. Hopefully tools like this can stop image AI dead in it's tracks.
Not exactly concerning image AI, but regarding LLMs -- in case your site does not depend on indexing -- you can easily generate boatloads of garbage text via Markov chains and invisible-link them.
Can you explain why we would want to stop image AI in its tracks? Why do we want to replace taxi drivers and factory workers but not cheap crap commercial artists?
Given that real art is both incredibly cheap to produce and incredibly cheap to distribute I should think those who are so inspired will continue to produce works to inspire us. I mean the advent of printers, cheap prints, and stock images didn't destroy art so why should AI?
Artists and photographers were never asked permission of their photos to be used to develop image AI yet they ended up doing so, it was built off theft and will continue to be build off theft by using user data from social media companies.
Ah, someone who doesn't understand how this stuff works and is just unnecessarily mad. People posted that stuff on the internet, it's not retaining their property it's learning the nuances of it. So it's effectively me looking through someone's gallery then attempting to create something like their art. No, it doesn't create "perfect recreations" or "perfect stylized things of theirs" because the outputs frequently are only stylistically similar. I'm sure you could get something that looks like one of their original pieces out with some effort prompting and conditioning the output but haven't people been copying artwork for millenia? It's definitely not a new phenomenon and preventing people from using generative art isn't going to stop it.
Think Alex Grey vs Cameron Gray. Both amazing artists but looking at Cameron's work he clearly is inspired by Alex. Did he steal that? Does he deserve to be shutdown? Maybe he asked permission but is Alex Grey financially burdened by Cameron's existence?
Artists are just mad something can do cool stuff that allows people who haven't spent a decade practicing something to create what their minds imagine.
72 comments
[ 1.7 ms ] story [ 140 ms ] threadhttps://news.ycombinator.com/item?id=37990750
Even worse, we are prone to being influenced by the decor or surroundings in ways that we know, but still partake in. Mood lighting would be an easy example. And we are very influenced by our other senses at the same time as purely visual. (Though, I'm always amused by how much the VR experience of racing a car hits, all without too much haptic feedback from the steering wheel.)
Iirc there was also some license violation with the cloaking code ironically, it was like a pillow fight
Apparently this is the same team and they claim that the previous one was only intended to prevent training on an artist's style.
ML researcher culture is hard to describe... But its something like "Everything is a research prototype, so why does it matter?"
Source? The authors claim this is not the case, and they have experiments showing it.
You will never get courts to conflate the two issues because they are fundamentally different. I have a constitutional right to my speech. You do not.
But I'll humor you. Let's assume that they sue me. Know what I'm gonna do? I'm going to sue them for the value of their entire corporation for stealing my work.
All progress comes at labor from the past. If you didn't utilize past labor, you would not be talking to us now on the internet. You would be banging rocks against some cave wall instead.
If someone is inspired by art and they become an artist creating a similar style, is that stealing from you too? Is using a computer stealing from the engineers that created them? Is sleeping in a house stealing from the architects and builders?
If someone speaks do they need to pay royalties to the creators of the English language?
This technology is the beginning of creating a real life holodeck. Anything someone imagines they can creatively bring to life. How is it possible there are people greedy and self centered enough to not desire that for the future?
It is hard to follow the logic of your stance. I am genuinely curious about your modality of thinking if you would care to elaborate.
Generative AI is already allowing people to unleash their creativity in miraculous ways, thought impossible previously. I feel your argument is unfortunately extremely dated already. I am also not seeing your connection between holodeck and replicators to Amazon drivers peeing in bottles? I wonder what coal miners have to do to make a living? There are all kinds of grotesque jobs and practices, the aim is to eliminate all of them in time with automated labor, and taking all the necessary supportive actions to make that possible / sustainable. Generative AI distributes creative control and power to everyone instead of those with money to afford it.
You do not. So if you come to steal from me, do not be surprised when I poison the living shit out the well you're attempting to drink from.
The greed here is not from the content creators, but from the wannabe AI overlords who believe they have the right to profit off of other people's work without their knowledge or consent.
You want data to train your AI? Either create it, buy it elsewhere, or pay me according to MY terms. But you won't get it from me for free and I will purposefully make your life hell if you attempt to do so. I don't give a damn about whatever pie in the sky nonsense you can come up with about technology and advancements. Your desire to get rich in the future is not more important than my need to eat today.
This is absolutely correct. However, the tool in question attempts to poison the entire model, including the work you never created.
Not my fucking problem.
Looks like that type of art gets to be a passion project for people again instead of a way to make a living. Creative, passionate people will still create regardless of the machines. The machines will do all the soulless breadwinning artwork. I am and will continue to celebrate this change where raw and pure creative expression is accessible to more people.
If you don't want to adapt, I am sure your financial needs will necessitate getting a new skill you can still monetize, hopefully one that doesn't involve flipping burgers because of a hard-nosed stance (nothing wrong with burger flipping, it is just those jobs will be done by robots soon too).
I think it seems very likely that that this will always be the case: if it is visually neutral enough to be acceptable for use, there will be a way to train systems to generate art that visually resembles the original based on the output.
I think their point might be that we need to figure out how we live in a future with AI in it. Trying to deny AI with tools like this is not going to work.
You know, technological progress. We don't stop it when it affects accountants, delivery drivers, programmers, etc. Art should be no different.
This is good peak of inflated expectations material [1]. LLMs will wholly transform some industries. Others will be virtually unchanged.
[1] https://en.m.wikipedia.org/wiki/Gartner_hype_cycle
Jerks are going to find dumb, lazy, malicious ways to abuse good tools, like they always do, though diffusion does have an conspicuously low effort barrier.
Another HN user summarized my thoughts on this better than I possibly could: https://news.ycombinator.com/item?id=37978805
And that's was some actual AI artists do too. Its massively turbocharging stuff thats they can do in an illustrator or photo editor, but just takes forever. The tooling around diffusion is a deep rabbit hole, and requires some skill to use.
There's an important distinction between that kind of artist and the "AI users" flooding the internet. They are more akin to dank meme makers using Photoshop.
Being forced into an "average" grade sounds okay, but rather, I think, what is happening is that a new baseline for minimum meaningful art was created, and at the same time, new definition for value of art is being established as errors from that baseline. And I'm not sure those are good things.
I'm not saying that we will soon have a state-of-the-art image generation model that relies on self training, supervised learning or adversarial learning rather than a scraped data set, but I think it's a highly likely outcome within the next few years. Anyone making the argument that AI art is just copying might be in for a rude awakening. If that comes to pass, will you eagerly embrace AI art?
My experience with ESRGAN is that it really does not like to train on its own output. It amplifies all sorts of wierd artifacts you can barely see I'd you go back and squint at the finetuning dataset. And there was a paper suggesting newer generative models are similar.
Sometimes you just need some textures to make your game's level look pretty.
Whether you call that art or not is in the eye of the beholder. Compare https://en.wikipedia.org/wiki/Applied_arts
…
To quote an artist that I know, "artists already steal. They're the masters at it."
Stealing inspiration and style is how artists learn. It just takes time. They're angry that the time investment is going to zero. And also that their examples are what are being used, despite the fact that other human artists are doing that already.
I'm not sure we should defend any difficulty or complexity. Even that of our own careers.
We'd still be churning our own butter if we were afraid of automation. And what time would that leave us with?
The typical argument against this is that human artists and ML models learn the same way, but the problem is that there's a fundamental difference: humans produce art by abstracting the concepts (think directional lighting, the kinematics of skeletal creatures, color theory, or physics of cloth) from the pieces they've studied and combining that with what they've learned from observing the world around them and by producing original art. This is why a human artist can produce competent art after studying only a handful of art pieces and gain most of their skill through practice.
By contrast, ML models do not learn by producing and have no way to directly observe the world. They don't abstract concepts, they "learn" by memorizing huge numbers of pre-existing images, and they produce images by patching together tiny bits and pieces of of those memorized images.
Where the human artist can function on little to no input, an ML model cannot, and so it's only fair that human artists have the choice to not have their works included in training datasets because if there is no feedstock of art there is no AI generated imagery.
I have a clog you can throw into a loom if it'll make you feel better, but there's no stopping AI art.
Simply rescale all images in a preprocessing step.
Since this defeat is so easy, and so obvious, either the authors are daft, or they are brilliant and have thought of a way to prevent it.
So, not wanting to waste much time on a paper from daft people, I did a quick skim of the paper. I don't see any mention of this defeat. No prevention of it, no assessment of it, not even any evidence of awareness of it.
Did I miss something?
I'd hate to see technically naive artist types (a mere subset of artist types, I acknowledge) being hoodwinked by such snake oil and paying money for such "tools."
One could also submit each image to an image-to-image model with a blank prompt and the right settings to essentially ask for almost a copy, and come out with washed images. Could even do a sharpening (or upscale and sharpening for that matter) at the same time. It has a small (tiny really) GPU cost per image but it's a one-time cost.
So I think simply rescaling would be a lot easier.
Most likely though, it will be simple to strip out.
Seriously, there's a "copyright derangement syndrome" going on right now, and these people need to settle the fuck down already.
One cool project for that> https://github.com/jsvine/markovify
(Personally had the most fun outputs when constructing the corpus from technical articles mixed with esoteric babble)
Active defense!
Given that real art is both incredibly cheap to produce and incredibly cheap to distribute I should think those who are so inspired will continue to produce works to inspire us. I mean the advent of printers, cheap prints, and stock images didn't destroy art so why should AI?
Think Alex Grey vs Cameron Gray. Both amazing artists but looking at Cameron's work he clearly is inspired by Alex. Did he steal that? Does he deserve to be shutdown? Maybe he asked permission but is Alex Grey financially burdened by Cameron's existence?
Artists are just mad something can do cool stuff that allows people who haven't spent a decade practicing something to create what their minds imagine.