There's no stopping this technology, but we can do without all of fake "for the good of humanity" stuff. If you want free pictures in someone's style without paying an artist for them, just be honest.
I really dont think using his name in a prompt had as much to do with his style to some degree, but more that it had an unusual amount of influence on the resulting image compared to other artists. Like his name had an unusual weight to the model that made images higher quality. Images that are nowhere close to his style still had "by Greg Rutkowski" in it. in the same way they have "Masterpiece, Best Quality"
If Rutkowski didn't already exist, would AI artists ever be able to arrive at his style from first principles? It seems that all novelty entering the AI art ecosystem comes from riding the coattails of non-AI artists, they're literally incapable of producing, for example, fan-art of a newly revealed character until non-AI artists produce enough examples to train a LoRA on.
Months on from the release of the new Zelda game I still can't find any on-model AI art of most of the distinctive characters. They just can't do it without a LoRA, and you can't make a LoRA without many existing examples, so they never could have put those characters to paper using AI if the design only existed in their head. Conventional artists have to do it first so that AI artists can follow.
Artists using AI maybe could have. “AI artists” are largely a bunch of obnoxious trolls with a shallow understanding of art history, technique, etc, such that this seems unlikely. Almost all AI art I see is within a narrow range of stylized realism or anime/manga-derived work.
There are novel artists using Stable Diffusion or Firefly as a photoshop-like-tool. Not txt2img, but stuff like inpainting, controlnet, contextual modification, LORAs trained on their own portfolio and such.
They are obscure for the same reason most artists are obscure.
But most "AI artists" you are talking about are would-be consumers looking for something specific, not creators looking to pull something novel out of their head.
I don't look at that kind of artwork all that much, but Rutkowski's pictures seem to have a very D&D/Tolkien-ish quality to me, to put it in some words. I'm pretty sure other artists made works in similar styles.
I do recall purchasing a poster with a similar look around 20 years ago, when he'd have been 14-ish. He might be a prominent artist in the genre now, but he's far from the first or that unique.
I'm sure he's influential, but D&D and MTG were around before he was drawing, and he didn't develop his style in a vacuum. There's a lot of similar work out there.
Putting greg rutkowski in a prompt produces compositions that have a fantasy/sci fi edge, more muted earth tones, areas of both clean and rough brush strokes and areas of large contrast with a tendency towards a central focus. It also makes for better dragons/castles/etc. Nothing magic.
By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different?
A fan artist must observe a character's official depiction before they can produce their own variants of it. Most humans do not produce art in a clean room, either - we all produce a synthesis of the existing art we've been exposed to. A LoRA or dreambooth model incorporating a novel concept is conceptually similar to a fan artist being shown a new character so they can produce new derivations of it.
You could produce "police sketches" of a previously unseen character based on shared description, but it would likely fail to capture the nuance of the individual just as police sketches would.
> By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different?
I've yet to see a LoRA produce a meaningfully diverse and high quality range of interpretations based on just a single piece of character key art, or brief appearance in a trailer, the way the conventional fan-artists do. What actually happens is that those conventional artists produce a wide variety of interpretations, someone scrapes those hundreds or thousands of images, and then trains a model using that dataset instead. The model needs the variety and novelty that the human artists bring to the table, otherwise it would just over-fit to the official art.
When I was looking for examples of Zelda AI art I noticed there is a LoRA for Purah, and the first version was posted on CivetAI not when her TOTK design was revealed, nor when the game came out, but a month later when there was a wide enough corpus of existing fanart to scrape together a decent model with. It only works by piggybacking off the non-AI artists effort.
I get what you mean, but my point is that fundamentally, what's happening in the LoRA and in the human artist is conceptually similar - each learns through examples what it is about a character that makes that character distinctive, and then attempts to reproduce it within some set of constraints. Neither is capable of doing so without first being exposed to what it is that makes that character distinctive. Human brains are obviously far better at interpretation than current AIs are (and have the advantage of actual will and intent), but I don't know why one'd expect a model that's never seen a character to be able to reproduce the essential nature of that character.
I rather suspect that you can get more variation than you'd think from a homogenous dataset, though - I've trained a LoRA on myself that can produce photorealistic renderings of me, as well as anime, Disney, Pixar, and Dreamworks styles, all of which actually feel pretty "me"; I don't have any human-produced drawings of myself as a Disney character or renderings of me in a Pixar movie, but the model does a good job of projecting what I'd look like given what it knows about "Disney-style" drawings, plus what I've taught it about what I look like.
For fun, I took two shots of Sonia from the third TOTK trailer (who should be novel and not in the model), trained a LoRA for 400 steps on just them (based on the AbyssOrange anime model), then threw it into an entirely different model (Dreamshaper, which mixes anime and realistic models) with some prompting to get something other than the original.
I'm not gonna say that it's perfect, but as an interpretation of the character based on exactly two nearly-identical screen grabs trained for a whole 400 steps and some amateur prompting? Not too shabby at all. It hit many of the distinctive points of the character - darker skin, blonde hair with ringlets, the forehead emblem, the tear tattoos under the eyes, even the laurels under the hair. It missed the bell earrings and the secret stone necklace, but given the limited source material, I think it's quite good. With a few more varied screenshots, I'll bet that it could do quite impressively.
Rutkowski was /never/ in Stable Diffusion - the article is incorrect.
His name worked in the text encoder released by OpenAI, but it was more like a synonym for "concept artist". The image encoder didn't see any of his images.
So if he's not in a newer text encoder, you could find him again with textual inversion.
> “If he contacts me asking for removal, I'll remove this.” Lykon said.
My interpretation of the article is that the entire reason this person created this was because the artist had objected to being in a generative model. When the artist was removed from the popular generative model, this person created a model/tuning specifically to put the artist back into one, and distributed it. They seemed to do this well aware that the artist didn't want to be in a generative model.
That sounds aggressively adversarial and abusive, and so the quote above sounds disingenuous.
Consider the analogy of a bully who is hitting a victim, and the bully demands that the victim ask the bully nicely to please stop. Humiliating the victim, and making the victim acknowledge the bully's power over them. Also, this is only after much damage is already done to the victim. And, in this case, after the bully has called over a mob to also beat the victim, and knowing that, even if the bully stops, the mob will continue.
The person in this case might not have the self-image of a bully, but is instead thinking of it as the usual Internet get-out-of-jail-free card: such as when you post someone else's content, when you're pretty sure they don't want you to, and without even trying to get permission, but you think it's all OK if you say "I'll take this down if they ask me to". (Related: The YouTube copyright violation disclaimer "I don't own this! All rights to the owner!", maybe even referencing the "All rights reserved" of a traditional copyright notice.)
Wherever you stand philosophically on copyright and related matters, and whatever jurisdiction and economic conditions you're under, it's maybe also helpful to consider the bullying question. One can stray into that accidentally, and that's harder to justify.
If this person was genuine, they would have asked permission first. It wasn't like they had to ask a lot of people who are hard to contact. Just 1 person.
I'd have had more respect for this person if they'd just said "Sorry not sorry." instead.
The thing is, most styles don't have names, only artists using them. If you remove artist names from the training data, you lose the ability to describe those styles, including as combinations (so you can't even do "influenced by rutkowski" styles). This obviously sucks for the user, and I don't think artists should be able to deny control over style just because it makes imitation easy.
In this context, putting an artist back in the model is a pretty sensible form of disagreement/protest. I admit the "I'll take this down if they ask me to" is strange, but mostly because its too weak; the LoRA should just stay up, not demand another no from someone who already made their opinion clear.
> If you remove artist names from the training data, you lose the ability to describe those styles
It's almost as if artists develop their own styles that are just as uniquely theirs as any one work produced by them in that style, if not more so! When people put "by Greg Rutkowski" in their AI art prompts, they are not asking for results in a style that exists independently of Greg Rutkowski despite his name somehow being the only way to identify it. They are, in fact, asking for results in the style of Greg Rutkowski in the sense that works by Greg Rutkowski were labeled as such and fed into the training process, programmatically creating a digest of works by (at least) Greg Rutkowski that we call a "model", and that model is as a result capable of mimicking the style of Greg Rutkowski specifically. They are asking for results that look like they were painted by Greg Rutkowski, and they care enough to ask for Greg Rutkowski specifically because Greg Rutkowski is a highly skilled artist who developed his own distinctive style which is the central pillar of his intellectual property as an artist, and his competitive advantage.
Intellectual property law as it stands today obviously couldn't have anticipated this technology, and didn't. However, if this kind of thing is still legal in ten years, that will represent a major systemic failure of copyright and trade secrets law to protect the core intellectual property of artists in the same spirit as it currently protects intellectual property in other domains.
> They are asking for results that look like they were painted by Greg Rutkowski, and they care enough to ask for Greg Rutkowski specifically because Greg Rutkowski is a highly skilled artist who developed his own distinctive style
Sometimes, but not always.
Lots of people prompt by cargo cult -- you find lots and lots of images generated by peppering the list with stuff like "masterpiece, best quality, 8k, award-winning".
Are all of those found in the training set? Do they actually have the right result?
It's statistics, not magic. The machine doesn't know what you mean, it only knows what's in the dataset. Some tags are not well represented enough to have the result you'd expect from the name.
Googling around suggests "masterpiece" mostly translates to higher contrast and saturation.
> they are not asking for results in a style that exists independently of Greg Rutkowski despite his name somehow being the only way to identify it.
Some might be. There's not always a tag for every kind of aesthetic, so one possible solution is to find an artist that uses it.
> which is the central pillar of his intellectual property as an artist, and his competitive advantage.
And which copyright law says he doesn't have. Style isn't copyrightable.
Sure—they often have such a poor understanding of art and the artist that they may not know that's what they're looking for, but that doesn't change the fact that it is, indeed, what they are looking for.
> And which copyright law says he doesn't have. Style isn't copyrightable.
The part of my comment you quoted says nothing about copyright law, so I'm not sure what you're trying to contradict here. An artist's personal style is a central pillar of their intellectual property in the sense that it takes massive personal investment to develop and everything they create flows from it, and it is a central pillar of their competitive advantage in the sense that it's aesthetically unique, positively stimulating to the viewer, and links their identity to their entire body of work in a way that can be exploited in the marketplace.
The fact is that copyright law currently has no idea what to make of these AI tools. I think the only concrete opinion so far adopted by the US copyright office (which may in the future be superseded by legal precedent or law) is that products of AI alone are not products of human authorship and thus cannot themselves be copyrighted. It will take time for arguments to be made, for money to change hands, and finally for more broadly useful law to be settled, which is why I previously alluded to a period of ten years to let things shake out.
If copyright law does step up to protect artists from this novel attack on their intellectual property, I doubt that protection will come in the form of allowing styles to be broadly copyrightable. From the contemporary commercial perspective, most of the value proposition of these tools is the ability to automatically and at scale launder the intellectual property of artists into derivative works that ostensibly do not infringe on the original works; given that, it is very much in the spirit of existing intellectual property law to write new law to protect artists.
Only time will tell whether the inevitable evolution of intellectual property law to account for AI tools favors individual artists or large tech corporations.
> Only time will tell whether the inevitable evolution of intellectual property law to account for AI tools favors individual artists or large tech corporations.
Definitely tech corporations. I'd be careful placing my hopes on copyright.
You know there's a lot of public domain out there, as well as permissively licensed images, right?
So here's my future prediction:
1. If forced, AI companies will prune their datasets of anything not permissively licensed.
2. They'll start friendly community initiatives to fill in the holes. Just like Wikipedia has requests for pictures, so can AI.
3. There's probably going to be more interest in doing more with less, and in training AI on AI output.
Given that AI output is deemed non-copyrightable, the end result is a vast amount of public domain pictures, which can be freely used without paying anyone.
Think it's unlikely? I think it'd very likely based on what happened with the software industry. The industry heavily encouraged Open Source as opposed to Free Software, and as a result today you can easily run a business with zero licensing concerns or payment to anyone.
And once you're there, there's no taking this back. There's not once in my life that I've thought "You know, it's nice that GCC and Clang are free, but I really feel like paying big bucks for a compiler".
> Definitely tech corporations. I'd be careful placing my hopes on copyright.
Yes, I expect you're right.
> [...] Think it's unlikely?
The part of your scenario I'm most skeptical of is the part where artists freely contribute their work to AI training sets en masse. Even putting aside the current (admittedly somewhat reactionary) trend of artists being loudly contemptuous of AI tools and (to a large extent) the people who use them, I don't think the incentives for artists to post their work online are quite the same incentives that drive programmers to open-source their work, and I really don't think having your work subsumed into a vast training set where it completely loses its identity is nearly as satisfying (or professionally enabling, for that matter) as being able to show e.g. that your open-source library gets 100k downloads every day and is a critical component of X, Y, and Z popular software packages. Art—especially art created for non-commercial reasons, which is what we're talking about here—is often very personal, and as a result artists crave direct engagement from the people looking at it.
If the hypothetical "community initiatives" you're proposing can make such contributions more rewarding, either by drastically improving attribution on the output side of the model (seems unlikely) or by offering some other incredible value proposition distinct from the "opportunity" to have one's works blended into the AI slurry, maybe. But if the AI industry and community ever do actually start listening to artists and respecting their intellectual property to the point that they are no longer simply taking what they want just because they can, I think it will be an uphill battle to convince artists to freely give the same without compensation.
It seems more likely to me that large AI companies would need to pay artists to contribute to training sets, making themselves and their peers partially or wholly redundant. I don't think that's great either, but at least it isn't bald-faced theft of intellectual property.
> The part of your scenario I'm most skeptical of is the part where artists freely contribute their work to AI training sets en masse.
You're not quite getting it.
First, there's the public domain. Centuries of art hanging in museums and the like is there for the taking. There's also some modern works there like I think works made for the US government.
Second, there's permissive licensing. For instance I uploaded a bunch of stuff to Flickr that I don't mind at all being used for AI. That's also free for the taking.
Third, if AI output is not copyrightable, AI output can be fed to itself. So lacks in SD can be filled by generating, filtering and reinforcing what you want.
Fourth, if any jurisdiction declares that training is fair game, that can also be added.
Fifth, it occurs to me that if AI output is not copyrightable, then it possibly allows proprietary AIs to be gradually blended in. Eg, say Adobe runs its own. It might own the training set, but if Adobe legally licensed everything, but the output is not copyrighted, then you can feed it to another system still and get some benefit from Adobe's work that way.
This constitutes already a starting set that can be used.
> But if the AI industry and community ever do actually start listening to artists and respecting their intellectual property to the point that they are no longer simply taking what they want just because they can, I think it will be an uphill battle to convince artists to freely give the same without compensation.
So you don't. You don't appeal to traditional artists, you appeal to AI users, who are seeing their cool tools attacked by the artists. You convince them to find more material in a public library, or to make some (since AI output is not copyrighted).
> I don't think that's great either, but at least it isn't bald-faced theft of intellectual property.
I honestly fail to see the long term benefit. I see little satisfaction in "Well, the machine took my job, but at least I fought a legal battle to make sure modern AI is trained from squeaky clean 18th century art".
Once you have a clean AI that's it, that's the end game. You can even sell that as a perk. Why deal with copyright, royalties, and all that nonsense? Here's a machine that will do whatever you want for cheap and won't ever ask for anything.
I guess I must be failing to respond to the point you're trying to make in the way you expect. I admit that I am not quite sure what that point is.
There are several overlapping concerns here. One concern is the possibility of AI systems violating the intellectual property rights of living artists whose work is not in the public domain and who have not given permission for it to be used in this way. Another (but not the only other) concern is working artists being outcompeted by AI in the marketplace. You seem to be responding to the second concern; I am responding to the first, since it's most relevant to the context of Greg Rutkowski's name and work being included in AI art training sets without his consent and (now) explicitly against his wishes.
If you think I'm arguing that removing copyrighted works from the training sets of AI systems whose operators haven't secured the rights to said works will ultimately prevent such systems from partially or totally outcompeting human artist(s) in some market(s), you're wrong. I have not argued that and I will not, since I don't believe it. What I believe is that the cat is out of the bag with these systems and that we will be stuck with them for the foreseeable future, regardless of how their creators manage to finagle training sets, regardless of their impacts on the prospects of working artists and more broadly on culture. I suspect attempts to curtail their use or ban them outright would do more harm than good. But I also believe that all of the above being true doesn't give us the right to run roughshod over the existing intellectual property rights of artists as we grease ourselves up for the long slide into cultural oblivion.
> Only time will tell whether the inevitable evolution of intellectual property law to account for AI tools favors individual artists or large tech corporations.
In the end, its going to be “large tech corporations” vs. “large media corporations”, with both sides citing individual artists (in one case, their users, and in the other, their suppliers) to present the clash of titans as a David vs. Goliath story with themselves cast as David.
That list is only really useful for major styles that are well known/tagged, early SD versions knew a list of artists _much_ longer and more relevant than that. (some of the styles on that list are not related to 2d images). If you consider the list a reasonable substitute for artist names, I'd ask you for a prompt that replicates the common Rutkowski + Mucha style using just the styles listed.
>> It's almost as if artists develop their own styles that are just as uniquely theirs
Hard disagree on this. Styles do not and should not have ownership, legally or otherwise. Not in painting, music, film, games, or any other kind of art. Anything else kneecaps derivative styles and erects a massive barrier to the advancement of culture. Straight copies are mostly avoided because they add nothing over the original.
You're also misunderstanding the typical use of Rutkowski and other names. Most prompts I've seen and used use multiple artists for a unique blend of styles (including the much maligned and overused Rutkowski + Mucha), not just straight copies. While asking for Rutkowski indeed asks for a very specific style (as well as any themes etc common to his work), its usually in combination with other equally specific and somewhat contradictory styles, each guiding the result in some direction and adding elements but none of them being exactly the desired end result. Just asking for straight up Rutkowski was a short lived cargo cult and is now more of a meme than anything else.
> If you consider the list a reasonable substitute for artist names
I didn't say or suggest any such thing.
> I'd ask you for a prompt that replicates the common Rutkowski + Mucha style using just the styles listed.
You're completely missing the point, which is that if you want the model to (e.g.) combine the individual styles of two specific artists in its output then you are not just looking for a style, you are looking for mimicries of those specific artists combined and normalized by the model. The model is able to do this because works by these artists were annotated as such and fed verbatim into the training process. Of course it can't mimic a specific artist unless you mention them by name!
> Hard disagree on this. Styles do not and should not have ownership, legally or otherwise. Not in painting, music, film, games, or any other kind of art. Anything else kneecaps derivative styles and erects a massive barrier to the advancement of culture. Straight copies are mostly avoided because they add nothing over the original.
You (the general you, an amalgam of the AI art defenders I've seen) claim the right to create these unauthorized derivative works via an automated process with zero reliance on novel human authorship beyond a short text string, then work backwards from there to rationalize to yourself why what you're doing is okay. Then, when artists who've invested decades of their lives into the development of their technique and process and relationship with art tell you that your rationalization is a facile one and that you are stealing from them, you are so sure you know better than them that you sweep aside their concerns and accuse them of jealousy.
In context, it's an absurd and frankly disgusting level of self-absorption. It's like the way crackpots relate to real working scientists, angrily casting the lack of attention paid to them and their "theories" as jealous conspiracy and gatekeeping rather than considering that maybe they simply have no idea what they're talking about, because why would they? But in this case the big money is unfortunately on the side of the crackpots.
I don't care about the "meta", and I don't care about "memes" or any other specifics of the subculture of AI art tool users. What I care about is that these tools obviously launder the intellectual property of artists into ostensibly non-infringing derivative works. I think the comparison with how human artists develop and apply technique and style is intellectually fascinating and worthy of discussion^[1], but from a legal and moral perspective comparing the two is like comparing a detective listening to a wiretap and taking notes by hand with the NSA's PRISM program or similar. They are superficially the same from a certain narrow perspective (and I believe there has been no end of facile rationalizations deployed to this effect), but broadly they are substantively different, morally and practically and especially in their implications for society. Everybody knows that justifying the latter on the basis of the former is foot-in-the-door bullshit that flies in the face of the spirit of the law.
Where will the law go? Who knows! All I know for sure is which side has the money.
^[1] I think if you go back in my commenting history you'll see me talking about that at length, and even taking what may appear to be the other side of this debate.
> You (the general you, an amalgam of the AI art defenders I’ve seen) claim the right to create these unauthorized derivative works via an automated process with zero reliance on novel human authorship
If they are created “via an automated process with zero reliance on novel human authorship” they, ipso facto, are not “derivative works”, which are original creations of human authorship subject to separate copyright (which, nonetheless, require permission from the copyright holder of the original work to legally create unless an exception to copyright protection, like fair use, applies.)
What you probably want to argue is that they are unauthorized mechanical copies, not unauthorized derivative works.
Of course, if they are unauthorized mechanical copies, it is probably usually a partial copy of the training set, which if it has a copyright it is a compilation copyright owned by whoever assembled the training set, not of any particular work within the training set. Now, whether the training set is an unauthorized derivative work when compiled is another question.
I don't really want to argue either because I'm not a lawyer and from a lay perspective AI works don't seem to fit neatly into either category. It's obvious to me that infringement occurs in spirit, which is effectively what i'm arguing; beyond that I really couldn't speculate on the details of how a successful case to that effect would be built, or how new law would be written.
> If you consider the list a reasonable substitute for artist names, I’d ask you for a prompt that replicates the common Rutkowski + Mucha style using just the styles listed.
Not looking at the list, but replacing “by Greg Rutkowski” with “detailed realistic muted fantasy” and “by Alphonse Mucha” with “Art Nouveau” seems to work pretty well for reproducing both “by Greg Rutkowski” alone and for the Rutkowski + Mucha combination. (Oddly, replacing either of those, but not the other, does not work as well for reproducing the combination effect as replacing both.)
> Just asking for straight up Rutkowski was a short lived cargo cult and is now more of a meme than anything else.
Rutkowski + Mucha is as much of a cargo cult as Rutkowski alone, even if it is a longer-lived one.
> If you remove artist names from the training data, you lose the ability to describe those styles
Do you?
With base SD1.5, “by greg rutkowski” produces extremely similar results to “detailed realistic muted fantasy”. Except for the case where what you are actually trying to do is directly mimic the work of an artist with a strong unique style that is very heavily represented in the source data (Rutkowski, IIRC, has extremely little representation in the SD training data, leaving aside question of whether the other parts of that description apply), I don’t think you are losing the ability to refer to styles without artist names.
I think Civitai, where the Rutkowski LoRa is hosted and where the screenshot in the article is from, in part set the tone during the earlier conflict with Sam Yang/@samdoesarts. I think Yang posted about someone on r/StableDiffusion making a LoRa trained on his work, which prompted Civitai to host a contest to see who could make the best samdoesarts-model. And send him a mail:
https://files.catbox.moe/adyb0x.png
A digital quagmire where Rutkowski himself struggled to differentiate between AI-generated pieces and his genuine works.
Is this hyperbole or did he actually say this? How could someone spend tens of hours meticulously painting something and then not be able to tell if he did it?
Nonsense. A neural net of sufficiently high parameter count can learn a function to reproduce any digital representation of information. This argument would suggest that anything which can be represented digitally is therefore of minimal value.
> This argument would suggest that anything which can be represented digitally is therefore of minimal value.
Uh, not really like that. The original was about value in comparison, not an absolute value. I don't really see what's immediately wrong with an idea that anything digital has questionable value compared to its clones generated by a trained neural network (but absolute value of those could be still very high). It could be wrong - but it's not immediately obvious to me.
I suppose that means that he's having a hard time explaining to how to distinguish the AI version from his works. Like there's nothing along the lines of "This is clearly not my work because I don't draw dragon wings this way" he can say.
Not that he forgot what he drew and what he didn't.
That reads to me as "he looked at the details of the AI art, and had a hard time finding obvious issue in the 'technique'", not that he forgot about art he made.
Man I feel for the artists. Seen a lot of outcry among people I've worked with too. It just seems so unfair. Like what are we doing? We automated away one of the most pleasing, satisfing and skill intensive endeavours humans can undertake. A dreadful reality we'll probably have coming to our jobs soon enough too.
Bitter truth is though, it's not going away. The genie is out of the bottle, and if you are a talented artist, you absolutly need to embrace AI. If you take a look at DeviantArt AI section for example you'll see: The bar has been raised. It's masterpiece next to masterpiece now. Talented artists + AI = Simply stunning work.
It feels heartless, but what can we do, will you be able to stay competive without AI? It feels you'll be just hampering yourself for idealistics reasons.
We didn't automate away art. We automated away the technical skill of digital illustration, which a totally different thing. Now, instead of artists also having to be illustrators, we've opened up being an artist to a wider audience.
People know this but it doesn’t make things better. If you dedicated thousands of hours to mastering technical skill of illustration, it isn’t nice to see the value in that work obliterated overnight.
What makes this so much worse than things like chess AI is that chess is done for fun and the final output is useless. While art is mostly for the output. While the process is fun, it’s fun because of the value of the output.
> we've opened up being an artist to a wider audience.
No, we've opened up the ability for someone who writes a short textual description of an illustration to obtain an illustration ostensibly matching that description for a much lower cost than they might have paid for it before these technologies were available, e.g. by commissioning an artist.
You're right that technique can be abstractly separated from other aspects of the artistic process, but you miss how the inability to guide the application of technique will cut your capacity for expression off at the knees. The prompt is the art, such as it is, and the ability to iterate on versions of the prompt and parameters of the model to random-walk your way to a result that approximates what you were imagining is a very poor substitute for the ability to lay down the brush strokes yourself.
You can guide the application. We already have ControlNet, and tools like that are just in their infancy. By the time this stuff hits maturity tools like img2img and ControlNet will seem about as advanced as MS Paint vs Photoshop.
> No, we've opened up the ability for someone who writes a short textual description of an illustration to obtain an illustration ostensibly matching that description
Yes, that is the new art form which has been invented and opened up to a wider audience. By analogy, some people used to complain that DJs were not real musicians, because they just play records made by other people. While technically true, this observation is not interesting, because selecting and mixing records to create a larger sonic flow is its own art form.
I have no problem saying that a prompt to an AI system and subsequent iteration on that prompt, model parameters, etc. can be a form of artistic expression. However, the distinction between this form of artistic expression and the form practiced by the artist whose works the model was trained on is substantive and has important moral and legal implications, so when it is conspicuously absent from these discussions I do indeed find it "interesting" enough to make explicit.
While I do think many of them are stunning, there are quite a few that are obviously AI-generated with almost no human interference... Or perhaps none at all. The scenes don't make sense visually.
I think to be commercially competitive, you need to embrace AI. But to just do good art, you can still get there the old-fashioned way. It's similar to CNCs in that you can get a computer to do it for you, but you many, many people still do it completely by hand. And some people go half-and-half.
Right now it's not given that it's legal. Most people are assuming that it's ok under fair use, but until the SCOTUS rules on it it's an unknown.
Given the potential economic impact on the original artist, the mechanical nature of the transformation and the fact that extracting the style of an artist's work could be deemed substantive, fair use is not in the least bit a slam dunk.
One could argue that use of this sort of AI robs the original artist of the fruits of their labour, neutering copyright protections, and produces works that substitute for the originals, without adding any creative input other than commanding the output from a machine. Given the intent of copyright law, judges would likely be receptive to this argument.
60 comments
[ 3.3 ms ] story [ 226 ms ] threadSounds pretty unethical. :(
> “If he contacts me asking for removal, I'll remove this.” Lykon said.
... as if Greg hadn't taken a clear stance.
There's no stopping this technology, but we can do without all of fake "for the good of humanity" stuff. If you want free pictures in someone's style without paying an artist for them, just be honest.
Fun fact: we have yet to reach the one year anniversary of the initial announcement of Stable Diffusion [0]. A lot has happened in one year!
[0] https://stability.ai/blog/stable-diffusion-announcement
Months on from the release of the new Zelda game I still can't find any on-model AI art of most of the distinctive characters. They just can't do it without a LoRA, and you can't make a LoRA without many existing examples, so they never could have put those characters to paper using AI if the design only existed in their head. Conventional artists have to do it first so that AI artists can follow.
They are obscure for the same reason most artists are obscure.
But most "AI artists" you are talking about are would-be consumers looking for something specific, not creators looking to pull something novel out of their head.
I don't look at that kind of artwork all that much, but Rutkowski's pictures seem to have a very D&D/Tolkien-ish quality to me, to put it in some words. I'm pretty sure other artists made works in similar styles.
I do recall purchasing a poster with a similar look around 20 years ago, when he'd have been 14-ish. He might be a prominent artist in the genre now, but he's far from the first or that unique.
A fan artist must observe a character's official depiction before they can produce their own variants of it. Most humans do not produce art in a clean room, either - we all produce a synthesis of the existing art we've been exposed to. A LoRA or dreambooth model incorporating a novel concept is conceptually similar to a fan artist being shown a new character so they can produce new derivations of it.
You could produce "police sketches" of a previously unseen character based on shared description, but it would likely fail to capture the nuance of the individual just as police sketches would.
I've yet to see a LoRA produce a meaningfully diverse and high quality range of interpretations based on just a single piece of character key art, or brief appearance in a trailer, the way the conventional fan-artists do. What actually happens is that those conventional artists produce a wide variety of interpretations, someone scrapes those hundreds or thousands of images, and then trains a model using that dataset instead. The model needs the variety and novelty that the human artists bring to the table, otherwise it would just over-fit to the official art.
When I was looking for examples of Zelda AI art I noticed there is a LoRA for Purah, and the first version was posted on CivetAI not when her TOTK design was revealed, nor when the game came out, but a month later when there was a wide enough corpus of existing fanart to scrape together a decent model with. It only works by piggybacking off the non-AI artists effort.
I rather suspect that you can get more variation than you'd think from a homogenous dataset, though - I've trained a LoRA on myself that can produce photorealistic renderings of me, as well as anime, Disney, Pixar, and Dreamworks styles, all of which actually feel pretty "me"; I don't have any human-produced drawings of myself as a Disney character or renderings of me in a Pixar movie, but the model does a good job of projecting what I'd look like given what it knows about "Disney-style" drawings, plus what I've taught it about what I look like.
https://imgur.com/a/iPNPojK
I'm not gonna say that it's perfect, but as an interpretation of the character based on exactly two nearly-identical screen grabs trained for a whole 400 steps and some amateur prompting? Not too shabby at all. It hit many of the distinctive points of the character - darker skin, blonde hair with ringlets, the forehead emblem, the tear tattoos under the eyes, even the laurels under the hair. It missed the bell earrings and the secret stone necklace, but given the limited source material, I think it's quite good. With a few more varied screenshots, I'll bet that it could do quite impressively.
His name worked in the text encoder released by OpenAI, but it was more like a synonym for "concept artist". The image encoder didn't see any of his images.
So if he's not in a newer text encoder, you could find him again with textual inversion.
My interpretation of the article is that the entire reason this person created this was because the artist had objected to being in a generative model. When the artist was removed from the popular generative model, this person created a model/tuning specifically to put the artist back into one, and distributed it. They seemed to do this well aware that the artist didn't want to be in a generative model.
That sounds aggressively adversarial and abusive, and so the quote above sounds disingenuous.
Consider the analogy of a bully who is hitting a victim, and the bully demands that the victim ask the bully nicely to please stop. Humiliating the victim, and making the victim acknowledge the bully's power over them. Also, this is only after much damage is already done to the victim. And, in this case, after the bully has called over a mob to also beat the victim, and knowing that, even if the bully stops, the mob will continue.
The person in this case might not have the self-image of a bully, but is instead thinking of it as the usual Internet get-out-of-jail-free card: such as when you post someone else's content, when you're pretty sure they don't want you to, and without even trying to get permission, but you think it's all OK if you say "I'll take this down if they ask me to". (Related: The YouTube copyright violation disclaimer "I don't own this! All rights to the owner!", maybe even referencing the "All rights reserved" of a traditional copyright notice.)
Wherever you stand philosophically on copyright and related matters, and whatever jurisdiction and economic conditions you're under, it's maybe also helpful to consider the bullying question. One can stray into that accidentally, and that's harder to justify.
I'd have had more respect for this person if they'd just said "Sorry not sorry." instead.
In this context, putting an artist back in the model is a pretty sensible form of disagreement/protest. I admit the "I'll take this down if they ask me to" is strange, but mostly because its too weak; the LoRA should just stay up, not demand another no from someone who already made their opinion clear.
If you want to educate yourself on how to refer to different art styles you can start here: https://en.wikipedia.org/wiki/List_of_art_movements
> If you remove artist names from the training data, you lose the ability to describe those styles
It's almost as if artists develop their own styles that are just as uniquely theirs as any one work produced by them in that style, if not more so! When people put "by Greg Rutkowski" in their AI art prompts, they are not asking for results in a style that exists independently of Greg Rutkowski despite his name somehow being the only way to identify it. They are, in fact, asking for results in the style of Greg Rutkowski in the sense that works by Greg Rutkowski were labeled as such and fed into the training process, programmatically creating a digest of works by (at least) Greg Rutkowski that we call a "model", and that model is as a result capable of mimicking the style of Greg Rutkowski specifically. They are asking for results that look like they were painted by Greg Rutkowski, and they care enough to ask for Greg Rutkowski specifically because Greg Rutkowski is a highly skilled artist who developed his own distinctive style which is the central pillar of his intellectual property as an artist, and his competitive advantage.
Intellectual property law as it stands today obviously couldn't have anticipated this technology, and didn't. However, if this kind of thing is still legal in ten years, that will represent a major systemic failure of copyright and trade secrets law to protect the core intellectual property of artists in the same spirit as it currently protects intellectual property in other domains.
Sometimes, but not always.
Lots of people prompt by cargo cult -- you find lots and lots of images generated by peppering the list with stuff like "masterpiece, best quality, 8k, award-winning".
Are all of those found in the training set? Do they actually have the right result?
It's statistics, not magic. The machine doesn't know what you mean, it only knows what's in the dataset. Some tags are not well represented enough to have the result you'd expect from the name.
Googling around suggests "masterpiece" mostly translates to higher contrast and saturation.
> they are not asking for results in a style that exists independently of Greg Rutkowski despite his name somehow being the only way to identify it.
Some might be. There's not always a tag for every kind of aesthetic, so one possible solution is to find an artist that uses it.
> which is the central pillar of his intellectual property as an artist, and his competitive advantage.
And which copyright law says he doesn't have. Style isn't copyrightable.
Sure—they often have such a poor understanding of art and the artist that they may not know that's what they're looking for, but that doesn't change the fact that it is, indeed, what they are looking for.
> And which copyright law says he doesn't have. Style isn't copyrightable.
The part of my comment you quoted says nothing about copyright law, so I'm not sure what you're trying to contradict here. An artist's personal style is a central pillar of their intellectual property in the sense that it takes massive personal investment to develop and everything they create flows from it, and it is a central pillar of their competitive advantage in the sense that it's aesthetically unique, positively stimulating to the viewer, and links their identity to their entire body of work in a way that can be exploited in the marketplace.
The fact is that copyright law currently has no idea what to make of these AI tools. I think the only concrete opinion so far adopted by the US copyright office (which may in the future be superseded by legal precedent or law) is that products of AI alone are not products of human authorship and thus cannot themselves be copyrighted. It will take time for arguments to be made, for money to change hands, and finally for more broadly useful law to be settled, which is why I previously alluded to a period of ten years to let things shake out.
If copyright law does step up to protect artists from this novel attack on their intellectual property, I doubt that protection will come in the form of allowing styles to be broadly copyrightable. From the contemporary commercial perspective, most of the value proposition of these tools is the ability to automatically and at scale launder the intellectual property of artists into derivative works that ostensibly do not infringe on the original works; given that, it is very much in the spirit of existing intellectual property law to write new law to protect artists.
Only time will tell whether the inevitable evolution of intellectual property law to account for AI tools favors individual artists or large tech corporations.
Definitely tech corporations. I'd be careful placing my hopes on copyright.
You know there's a lot of public domain out there, as well as permissively licensed images, right?
So here's my future prediction:
1. If forced, AI companies will prune their datasets of anything not permissively licensed.
2. They'll start friendly community initiatives to fill in the holes. Just like Wikipedia has requests for pictures, so can AI.
3. There's probably going to be more interest in doing more with less, and in training AI on AI output.
Given that AI output is deemed non-copyrightable, the end result is a vast amount of public domain pictures, which can be freely used without paying anyone.
Think it's unlikely? I think it'd very likely based on what happened with the software industry. The industry heavily encouraged Open Source as opposed to Free Software, and as a result today you can easily run a business with zero licensing concerns or payment to anyone.
And once you're there, there's no taking this back. There's not once in my life that I've thought "You know, it's nice that GCC and Clang are free, but I really feel like paying big bucks for a compiler".
Yes, I expect you're right.
> [...] Think it's unlikely?
The part of your scenario I'm most skeptical of is the part where artists freely contribute their work to AI training sets en masse. Even putting aside the current (admittedly somewhat reactionary) trend of artists being loudly contemptuous of AI tools and (to a large extent) the people who use them, I don't think the incentives for artists to post their work online are quite the same incentives that drive programmers to open-source their work, and I really don't think having your work subsumed into a vast training set where it completely loses its identity is nearly as satisfying (or professionally enabling, for that matter) as being able to show e.g. that your open-source library gets 100k downloads every day and is a critical component of X, Y, and Z popular software packages. Art—especially art created for non-commercial reasons, which is what we're talking about here—is often very personal, and as a result artists crave direct engagement from the people looking at it.
If the hypothetical "community initiatives" you're proposing can make such contributions more rewarding, either by drastically improving attribution on the output side of the model (seems unlikely) or by offering some other incredible value proposition distinct from the "opportunity" to have one's works blended into the AI slurry, maybe. But if the AI industry and community ever do actually start listening to artists and respecting their intellectual property to the point that they are no longer simply taking what they want just because they can, I think it will be an uphill battle to convince artists to freely give the same without compensation.
It seems more likely to me that large AI companies would need to pay artists to contribute to training sets, making themselves and their peers partially or wholly redundant. I don't think that's great either, but at least it isn't bald-faced theft of intellectual property.
You're not quite getting it.
First, there's the public domain. Centuries of art hanging in museums and the like is there for the taking. There's also some modern works there like I think works made for the US government.
Second, there's permissive licensing. For instance I uploaded a bunch of stuff to Flickr that I don't mind at all being used for AI. That's also free for the taking.
Third, if AI output is not copyrightable, AI output can be fed to itself. So lacks in SD can be filled by generating, filtering and reinforcing what you want.
Fourth, if any jurisdiction declares that training is fair game, that can also be added.
Fifth, it occurs to me that if AI output is not copyrightable, then it possibly allows proprietary AIs to be gradually blended in. Eg, say Adobe runs its own. It might own the training set, but if Adobe legally licensed everything, but the output is not copyrighted, then you can feed it to another system still and get some benefit from Adobe's work that way.
This constitutes already a starting set that can be used.
> But if the AI industry and community ever do actually start listening to artists and respecting their intellectual property to the point that they are no longer simply taking what they want just because they can, I think it will be an uphill battle to convince artists to freely give the same without compensation.
So you don't. You don't appeal to traditional artists, you appeal to AI users, who are seeing their cool tools attacked by the artists. You convince them to find more material in a public library, or to make some (since AI output is not copyrighted).
> I don't think that's great either, but at least it isn't bald-faced theft of intellectual property.
I honestly fail to see the long term benefit. I see little satisfaction in "Well, the machine took my job, but at least I fought a legal battle to make sure modern AI is trained from squeaky clean 18th century art".
Once you have a clean AI that's it, that's the end game. You can even sell that as a perk. Why deal with copyright, royalties, and all that nonsense? Here's a machine that will do whatever you want for cheap and won't ever ask for anything.
I guess I must be failing to respond to the point you're trying to make in the way you expect. I admit that I am not quite sure what that point is.
There are several overlapping concerns here. One concern is the possibility of AI systems violating the intellectual property rights of living artists whose work is not in the public domain and who have not given permission for it to be used in this way. Another (but not the only other) concern is working artists being outcompeted by AI in the marketplace. You seem to be responding to the second concern; I am responding to the first, since it's most relevant to the context of Greg Rutkowski's name and work being included in AI art training sets without his consent and (now) explicitly against his wishes.
If you think I'm arguing that removing copyrighted works from the training sets of AI systems whose operators haven't secured the rights to said works will ultimately prevent such systems from partially or totally outcompeting human artist(s) in some market(s), you're wrong. I have not argued that and I will not, since I don't believe it. What I believe is that the cat is out of the bag with these systems and that we will be stuck with them for the foreseeable future, regardless of how their creators manage to finagle training sets, regardless of their impacts on the prospects of working artists and more broadly on culture. I suspect attempts to curtail their use or ban them outright would do more harm than good. But I also believe that all of the above being true doesn't give us the right to run roughshod over the existing intellectual property rights of artists as we grease ourselves up for the long slide into cultural oblivion.
I hope that helps to clarify my position.
In the end, its going to be “large tech corporations” vs. “large media corporations”, with both sides citing individual artists (in one case, their users, and in the other, their suppliers) to present the clash of titans as a David vs. Goliath story with themselves cast as David.
>> It's almost as if artists develop their own styles that are just as uniquely theirs
Hard disagree on this. Styles do not and should not have ownership, legally or otherwise. Not in painting, music, film, games, or any other kind of art. Anything else kneecaps derivative styles and erects a massive barrier to the advancement of culture. Straight copies are mostly avoided because they add nothing over the original.
You're also misunderstanding the typical use of Rutkowski and other names. Most prompts I've seen and used use multiple artists for a unique blend of styles (including the much maligned and overused Rutkowski + Mucha), not just straight copies. While asking for Rutkowski indeed asks for a very specific style (as well as any themes etc common to his work), its usually in combination with other equally specific and somewhat contradictory styles, each guiding the result in some direction and adding elements but none of them being exactly the desired end result. Just asking for straight up Rutkowski was a short lived cargo cult and is now more of a meme than anything else.
I didn't say or suggest any such thing.
> I'd ask you for a prompt that replicates the common Rutkowski + Mucha style using just the styles listed.
You're completely missing the point, which is that if you want the model to (e.g.) combine the individual styles of two specific artists in its output then you are not just looking for a style, you are looking for mimicries of those specific artists combined and normalized by the model. The model is able to do this because works by these artists were annotated as such and fed verbatim into the training process. Of course it can't mimic a specific artist unless you mention them by name!
> Hard disagree on this. Styles do not and should not have ownership, legally or otherwise. Not in painting, music, film, games, or any other kind of art. Anything else kneecaps derivative styles and erects a massive barrier to the advancement of culture. Straight copies are mostly avoided because they add nothing over the original.
You (the general you, an amalgam of the AI art defenders I've seen) claim the right to create these unauthorized derivative works via an automated process with zero reliance on novel human authorship beyond a short text string, then work backwards from there to rationalize to yourself why what you're doing is okay. Then, when artists who've invested decades of their lives into the development of their technique and process and relationship with art tell you that your rationalization is a facile one and that you are stealing from them, you are so sure you know better than them that you sweep aside their concerns and accuse them of jealousy.
In context, it's an absurd and frankly disgusting level of self-absorption. It's like the way crackpots relate to real working scientists, angrily casting the lack of attention paid to them and their "theories" as jealous conspiracy and gatekeeping rather than considering that maybe they simply have no idea what they're talking about, because why would they? But in this case the big money is unfortunately on the side of the crackpots.
I don't care about the "meta", and I don't care about "memes" or any other specifics of the subculture of AI art tool users. What I care about is that these tools obviously launder the intellectual property of artists into ostensibly non-infringing derivative works. I think the comparison with how human artists develop and apply technique and style is intellectually fascinating and worthy of discussion^[1], but from a legal and moral perspective comparing the two is like comparing a detective listening to a wiretap and taking notes by hand with the NSA's PRISM program or similar. They are superficially the same from a certain narrow perspective (and I believe there has been no end of facile rationalizations deployed to this effect), but broadly they are substantively different, morally and practically and especially in their implications for society. Everybody knows that justifying the latter on the basis of the former is foot-in-the-door bullshit that flies in the face of the spirit of the law.
Where will the law go? Who knows! All I know for sure is which side has the money.
^[1] I think if you go back in my commenting history you'll see me talking about that at length, and even taking what may appear to be the other side of this debate.
If they are created “via an automated process with zero reliance on novel human authorship” they, ipso facto, are not “derivative works”, which are original creations of human authorship subject to separate copyright (which, nonetheless, require permission from the copyright holder of the original work to legally create unless an exception to copyright protection, like fair use, applies.)
What you probably want to argue is that they are unauthorized mechanical copies, not unauthorized derivative works.
Of course, if they are unauthorized mechanical copies, it is probably usually a partial copy of the training set, which if it has a copyright it is a compilation copyright owned by whoever assembled the training set, not of any particular work within the training set. Now, whether the training set is an unauthorized derivative work when compiled is another question.
I don't really want to argue either because I'm not a lawyer and from a lay perspective AI works don't seem to fit neatly into either category. It's obvious to me that infringement occurs in spirit, which is effectively what i'm arguing; beyond that I really couldn't speculate on the details of how a successful case to that effect would be built, or how new law would be written.
Not looking at the list, but replacing “by Greg Rutkowski” with “detailed realistic muted fantasy” and “by Alphonse Mucha” with “Art Nouveau” seems to work pretty well for reproducing both “by Greg Rutkowski” alone and for the Rutkowski + Mucha combination. (Oddly, replacing either of those, but not the other, does not work as well for reproducing the combination effect as replacing both.)
> Just asking for straight up Rutkowski was a short lived cargo cult and is now more of a meme than anything else.
Rutkowski + Mucha is as much of a cargo cult as Rutkowski alone, even if it is a longer-lived one.
Do you?
With base SD1.5, “by greg rutkowski” produces extremely similar results to “detailed realistic muted fantasy”. Except for the case where what you are actually trying to do is directly mimic the work of an artist with a strong unique style that is very heavily represented in the source data (Rutkowski, IIRC, has extremely little representation in the SD training data, leaving aside question of whether the other parts of that description apply), I don’t think you are losing the ability to refer to styles without artist names.
Is this hyperbole or did he actually say this? How could someone spend tens of hours meticulously painting something and then not be able to tell if he did it?
Uh, not really like that. The original was about value in comparison, not an absolute value. I don't really see what's immediately wrong with an idea that anything digital has questionable value compared to its clones generated by a trained neural network (but absolute value of those could be still very high). It could be wrong - but it's not immediately obvious to me.
Not that he forgot what he drew and what he didn't.
Bitter truth is though, it's not going away. The genie is out of the bottle, and if you are a talented artist, you absolutly need to embrace AI. If you take a look at DeviantArt AI section for example you'll see: The bar has been raised. It's masterpiece next to masterpiece now. Talented artists + AI = Simply stunning work.
It feels heartless, but what can we do, will you be able to stay competive without AI? It feels you'll be just hampering yourself for idealistics reasons.
What makes this so much worse than things like chess AI is that chess is done for fun and the final output is useless. While art is mostly for the output. While the process is fun, it’s fun because of the value of the output.
No, we've opened up the ability for someone who writes a short textual description of an illustration to obtain an illustration ostensibly matching that description for a much lower cost than they might have paid for it before these technologies were available, e.g. by commissioning an artist.
You're right that technique can be abstractly separated from other aspects of the artistic process, but you miss how the inability to guide the application of technique will cut your capacity for expression off at the knees. The prompt is the art, such as it is, and the ability to iterate on versions of the prompt and parameters of the model to random-walk your way to a result that approximates what you were imagining is a very poor substitute for the ability to lay down the brush strokes yourself.
Yes, that is the new art form which has been invented and opened up to a wider audience. By analogy, some people used to complain that DJs were not real musicians, because they just play records made by other people. While technically true, this observation is not interesting, because selecting and mixing records to create a larger sonic flow is its own art form.
I think to be commercially competitive, you need to embrace AI. But to just do good art, you can still get there the old-fashioned way. It's similar to CNCs in that you can get a computer to do it for you, but you many, many people still do it completely by hand. And some people go half-and-half.
Right now it's not given that it's legal. Most people are assuming that it's ok under fair use, but until the SCOTUS rules on it it's an unknown.
Given the potential economic impact on the original artist, the mechanical nature of the transformation and the fact that extracting the style of an artist's work could be deemed substantive, fair use is not in the least bit a slam dunk.
One could argue that use of this sort of AI robs the original artist of the fruits of their labour, neutering copyright protections, and produces works that substitute for the originals, without adding any creative input other than commanding the output from a machine. Given the intent of copyright law, judges would likely be receptive to this argument.