> txt2img prompt crafting is a bit of an art in its own right, as getting the model to spit out what you want isn’t always trivial.
I don't find the output images from a lot of "AI" art to be artistically interesting because an "AI" made them. The output might be novel, or even kind of surprising, but still just the output of code. Someone might say the code of an "AI" art machine could qualify as art. I often find the txt2img prompts artistic in a way that the output images lack, because they are representative of human imagination, often like surreal poetry, or like musique concrete, cutting and pasting segments around on a tape. This is not to deny that the output of "AI" art is impressive. It is, frankly. But as a matter of machine, computational, output rather than human artistic merit.
It comes down to a simple thing: if you showed me two images and you told me a human made one, and "AI" made the other, I will always find the former artistic in a way that I would never find the latter.
This is also not to touch on the licensing issues that arise from these tools. That's its own knot to untangle.
There's a spectrum of human agency in art. Where will you draw the line? Where does Warhol fit? Pollock? Bridget Riley, Vasarely or even Malevich?
Whatever line you draw will leave several canonical figures on the wrong side of it. Maybe that's OK and you'll draw your line in the sand. Just don't pretend it's a non-controversial consensus.
I agree. Also, it's not that stable diffusion spontaneously generates art: someone has thought of a prompt, written it down, iterated, and finally decided that a given image was good enough to publish. Is stable diffusion an "artist" or is it a tool?
I find "what is art" discussions impossible to resolve and untangle from prejudices and biases. The least problematic answer I've found comes from John Carey, paraphrasing: "art is whatever someone has decided it to be". In other words, if I decide something is art, then it is.
The problem shifts slightly to a more interesting way to pose this question: why some art has more value than some other art (or even "does some art has more value than some other art")? Equally difficult to resolve but more prone to highlight prejudices and biases mentioned above.
>I agree. Also, it's not that stable diffusion spontaneously generates art: someone has thought of a prompt, written it down, iterated, and finally decided that a given image was good enough to publish. Is stable diffusion an "artist" or is it a tool?
I'm curious about the possibility of the commissioner role for this situation.
From my point of view, the AI is the artist and prompting an AI to produce a specific image is akin to commissioning an artwork.
I think we have moved beyond human agency, and creation of art is reduced to the simpler constituents, the roles of artist and (if there is one) commissioner. The request to make an artpiece can also come from a machine.
I think the same argument can be made of photography. A photographer does not "paint" or "create" the image. It points a machine to a place and presses a button with a finger. This machine does "the work" for her. Who is the artist? Are photographers just commisioners of images?
Capturing the moment is more than just pointing.
Look, I know the argument you're making, and it's certainly something to ponder on.
But the whole "writing prompt" thing has a different aim from "human-created" art. Whatever that means. But it's generally a small subset from the latter.
I'm fully aware that I'm stretching the argument and analogies. However, I find all of these expressions ("Capturing the moment", "has a different aim") vague and full of gaps. Maybe it's because I never fully got photography, in a way. The difference between "capturing the moment" and "writing prompt" is that the former has a more romantic feel to it, but let's not forget that some of the most well-known photographies were staged to look spontaneous. And suddenly photography is just an exercise of story-telling and technique (light, exposure, etc), which is not that different from "harnessing the algorithm" to do the same.
Also, we're comparing an art 150 years in the making (with its schools, philosophies, heroes) with one in its infancy.
Sure, and if your 4 year old son draws 2 sloppy stick figures representing him holding your hand, that's going to be valuable to you in a way a stranger's precocious 10-year-old child's drawing isn't (no matter how good it is).
But does that make it more artistic than something generated by AI which was coaxed out by a human applying their expertise?
Art Turing test: If you can't distinguish between human-made art and AI-made art, then there's nothing backing up your emotional bias.
It doesn't matter anymore that you emotionally value, or want to emotionally value, human-made art more than AI-made art, if you can't distinguish the two except based on other people's representations of how it was made.
Unless you watched a (non-augmented) human make a piece of art, you have no assurance that a human didn't touch up an AI generated art piece or simply curate a collection of fully AI-generated art.
ETA: I'm not trying to put down recognizable human-created art that's already been created, especially historically notable works. I'm not sure what all those replies are trying to get at. Of course there can continue to be an art-collecting market for human-made works made before the AI-art era.
I can’t tell if someone is selling me something made locally, or is just pawning something made in China as local, but I still try to support local because I believe the underlying knock on effects are greater.
Supporting local manufacturing helps to build resilience in the supply chain. Outsourcing everything to another country can be great for margins but it adds risk via external dependencies.
That is not how art works at all though. It's very difficult to distinguish a real van gogh from a fake van gogh. The real will still sell for millions and the fake will not.
If a copy of a Van Gogh is next to the original, and you can't which is the original, do they have equal artistic value?
If an artist who studied Van Gogh intensely made an original piece in Van Gogh's style, does it have equal artistic merit as one of Van Gogh's originals?
AI is just going to be another aspect of provenance, and if it's found out an art piece had AI used to make it, it won't be forgotten or ignored.
For me personally, it's more about appreciating the act and capabilities possible by a human being.
I find it similar to watching chess. AI/Chess engines will always be better than humans, but it's exciting to watch a human play to see what they can do.
I think OP understands there may not be a physical difference, but the provenance / "Colour of the Bits" is important to them; and that's a valid position to take (not the only valid position to take).
For example, a woolen blanket made by my mother is much, much dearer to me than an equivalent woolen blanket made in a factory.
I see it very likely that certified-provenance human-art may command high prices in the future; but also for physical part, it's pretty trivial and doesn't have to be so posh and fancy - anything from street art to art galleries and art collectives/workshops.
This is already the case for the most part. I don't think anyone expects AI to displace "name brand" art. Low quality and stock art will become profitless virtually overnight, while artists that already make good money for original works will likely continue to do so. Your blanket analogy is apt for that reason. Human made blankets are indeed more special and people do often pay higher prices for them, but the vast majority of blankets sold or in use were made by machines at this point.
> I don't think anyone expects AI to displace "name brand" art
The problem isn't for existing artists (except in terms of ethical issues)—it's for new/budding artists, who will have to contend with challenges to the authenticity of their work. How can they prove that they put their blood and sweat into making a piece of artwork by hand when an AI could've generated something equally passable?
> If you can't distinguish between human-made art and AI-made art
The art is always human-made. Missing attribution doesn't mean "machine made it". Humans developed the AI, ran the AI, created the training data, picked the non-horrible sample from the output, etc.
The woolen blanket is not a precise example: you like one more for sentimental reasons, but you agree that both are blankets. If you had said "I don't think that a factory-made woolen blanket is really a woolen blanket" the discussion would be different.
I have a project that involves publishing an image I generate and publish to an audience everyday. It's not high value art, but the process does often involve me thinking of a concept I want to communicate to my audience and working with the AI to produce something that communicates that concept.
When I started doing this, I was uncomfortable thinking of myself as the author of the image in any way. I'm okay with it now. There's some skill I am expressing, even if not much, and I am connecting with an audience.
I don't think for "art as art" purposes where we take some painting and say it's worth a million dollars and put it in a museum ai can replace humans. But a lot of art is much more utilitarian: art in games, art in movies and tv, in picture books, etc. In those cases I care that it looks how it is meant to and costs as little as possible to make.
I feel like the majority of "art" we consume falls in the second category.
As a historical note, this argument is over 50 years old. In 1965, an engineer at Bell Labs generated pseudo-random artworks inspired by famous artist Piet Mondrian. He found that a) people couldn't tell if the artwork was generated by Mondrian or the computer, and b) people generally liked the computer's art better.
This argument could be made 150 years ago about photography. For the sake of an argument: can a selfie be a form of art?
To take a selfie, human intervention is minimal: someone decides they want to take a selfie; take dozens of them, pretty much at random; choose one they like; publish it.
Similarities with the stable diffusion process are striking. And yet, could you consider a selfie "artistic"? Are all selfies artistic? What if Annie Leibovitz or Marina Abramovic take a selfie?
What if Marina Abramovic generates a million images of a rabbit with stable diffusion and papers a whole warehouse with them? Is this art?
It's considered art for the purposes of e.g. copyright law. Recently there was a dispute about whether a selfie taken by a monkey belonged to the owner of the camera, or the monkey. (A judge opined that only humans can have copyrights, and PETA eventually settled out of court.)
If Marina Abramovic were to take a bunch of selfies, or generate a bunch of AI images, and exhibit them, that might be considered high value art. A famous artist can literally tape a banana to a wall and it'd be considered high art:
> Stable Diffusion? Well, that’s free-ish. Problem is, you’ll probably want to run it locally, which requires a really, really beefy graphics card. I was struggling to run it on a Vega56 - a GPU that goes for ~$150 used now - so I went out and got a RTX3090 for about $1,000. If you’re already a gamer with a GPU with 8Gb+ of VRAM you’re probably good, but for most people this is a bit absurd.
Nit: 2 weeks ago there was an article about getting SD running on an iPhone. The app isn't very time- or battery-efficient, but it just barely works on my (previous-gen) device: https://news.ycombinator.com/item?id=33539192
I get results in 2 minutes on my iPhone SE. At least, when it doesn't stop execution part way through. When that happens, I have to change the seed, but it happens often enough that it sometimes needs a second change of seed to actually complete.
Still, 2 minutes on my phone. I remember Bryce taking half an hour to render lower resolution images back in the late 90s.
Running on an iPhone-grade hardware is what they refer to as "struggling". You need a lot of iterations in any workflow, be it style transfer or text-to-image, so you want as fast generation and as much CUDA cores as possible, and up to 24GB VRAM for higher resolutions. More importantly, most of the power of this stuff is not just in inference, but in finetuning the model on your custom data, which requires 12GB VRAM for SD 1.4 at the very least, and ultimately multiple server-grade GPUs. And even more importantly, you need full control over your own box, not an iPhone app - as it's still a very experimental field which is only going to be more complex and hard to master.
That said, you can just rent a GPU instead of buying it right away. It's not very expensive, and can be cheaper overall depending on your usage. It's also the only sane way to experiment with training as you need a lot of attempts.
This stuff can really devour any hardware you can possibly throw at it.
> you can just rent a GPU instead of buying it right away
I agree, this is an underrated option and I'm not sure why it doesn't get more attention. I've been using stable diffusion on a cloud machine, and, say $20 will get you very far - probably thousands of images generated and a few sessions of fine tuning. It's way more affordable that its apparent popularity would suggest
This tech is currently barely usable for any real work outside of concepts, but it is easy to envision the development of new powerful tools built on top of refined iterations of the same models and algorithms, with finer controls and better integration.
Artists should rejoice, in my opinion, mastering this type of tool could dramatically increase their productivity, and thus their salaries.
I'm not sure it works like that. There's not an unlimited market for "things that look like original paintings". Demand won't scale with supply and "and thus their salaries" seems to ignore a couple of iron laws of economics.
However - I do wonder if this will increase the value of non-digital arts and crafts. They can't easily be substituted for - which is probably one of the key factors in establishing value in the art market.
> This tech is currently barely usable for any real work outside of concepts
I take it you haven't been keeping up. Stable Diffusion 2 and Midjourney are getting really, really impressive. Probably a 10x increase in quality over the original DALL-E already.
Now wait a few (years?) until this technology can generate 3d models, environments, lighting, animation, etc. You won't even need animators, just feed the script in and get rendered scenes with virtual acting, dialogue, score, everything out.
Everyone who's dismissing this because it imitates the style of another artist (as if that was a hard technical limitation rather than a specific prompt) or lacks the je nais se quoi of "real" art is missing the point. No one would ever hire a concept artist capable of doing this when they can hire someone for a fraction of the salary to write prompts that return acceptable results in a fraction of the time.
The difference is that it can effectively turn someone with very little artistic training into an artist. Sure, it's not usable for every use case, but it would be usable for album cover, or just to have nice illustrations to draw people to a website, and some cases of concept art.
Art and entertainment hinges on novelty and message. If you have too much of something, eventually it becomes boring, with or without AI. Until we have something that could pass for AGI, you're going to need people making something meaningful and novel for you, regardless of the sort of their technical skills they currently happen to have.
>Art and entertainment hinges on novelty and message.
Do they? Show me something truly novel either in terms of art or message created by Hollywood in the last decade. Art and entertainment are risk averse industries - novelty is risk, genre is safe, and within genre you have well defined aesthetics, themes and material from which to derive new iterations on a theme.
The mashups in the blog posted upthread (particularly the Giger/Henson posts[0]) look far more visually striking to me than most modern sci-fi. I want to see that movie based on the look alone, and the look is literally just "what if Dark Crystal, but xenomorphs?"
That's still "barely useable for any real work outside of concepts". It makes for some impressive thumbnails, but it still has all the obvious signs of an AI-generated image. This is despite it being a situation that's heavily playing into the AI's strengths by using abstract or fantastical content where departures from reality and mistakes of scaling or logical arrangement of elements are less obvious. The lighting is also something that's really helping the AI there, as the highly unnatural lighting will cover up any illogical shadows or highlights. Everyone's hands are all mutilated, it has real trouble with text, symmetry, circles, multiple instances of the same object, and if the scene has more than one subject, everything starts to get all melty.
I have tried all the new toys and I am keeping track of this fascinating development.
I work in the video games "industry", we need concepts and mockups, but a much larger part of the production is assets, meshes, textures, materials, shaders...
We're seeing huge improvements, from photogrammetry to simplified production pipelines and more powerful light models.
ML based tools will also help to iterate faster, but the key point is control, human control.
I'm keeping up with this pretty well, and it's very far away from being practical and controllable enough to get the actual job done. Txt2img is largely irrelevant for the real work, generation from text alone isn't going to work well beyond toy and novelty purposes.
I think you’re missing out a little on how fast this is improving and what exactly is possible already, especially if you have a powerful graphics card. Stable Diffusion has barely been out for three months and I am training my own custom models from my desktop for generating photos featuring my friends which would fool their families.
At first yes, faces and hands weren’t perfect but in the recent models and especially with model blending I can now generate super realistic photos of humans with perfect faces and eyes and perfect hands. Basically what is possible and the quality is rocketing forward right now and I can’t even begin to imagine where it will be in a year.
I would also add, the advances in prompt engineering, in-painting and image to image generation mean getting exactly the result you want with the composition you want is also very possible. If you’ve only generated a few images then you really have no idea of all of the tools and options you have. Over the last few months I’ve probably made 5000-10000 images and my own personal skill level for being able to get what I want has gone up massively.
>Artists should rejoice, in my opinion, mastering this type of tool could dramatically increase their productivity, and thus their salaries.
Salaries do not increase with productivity. Rather, they tend to remain constant or even decrease as productivity rises, as technology makes that level of productivity easier to hire for, and thus less costly for the employer.
"you’re probably not going to be using any of the AI art tools to make a reference sheet for a character, nor be able to get it to generate images that consistently have the appearance of a custom character"
This was true maybe a month ago, but DreamBooth makes this possible
Another recent development: Midjourney v4 has much better "image prompt" abilities allowing you to feed in various images of a character you have made to allow further scenes to maintain the characters likeness throughout.
I've read people making comics and generate many different scenes say v4 now allows consistent character design.
Can anyone with access to it test it out on the specific example in the post (fursona ref sheets), and show examples? If it is actually able to generate those with high accuracy that would seem to be a unbelievable improvement.
Haven't tested it on that, but I did try it with an anime character who might be somewhat similar: Yoshida Yuuko, aka. Shamiko. Given that she's a very specific sort of demon girl, it was completely impossible to get anywhere close to her appearance with any pre-existing model.
I took ten high-quality pictures off Pixiv, spent five minutes cropping them, and ran Dreambooth overnight on a 3090 — this turned out to be enormous overkill, the final model was massively overtrained. Intermediate ones were fine.
The output is perfect; even higher quality than what the input model was already good at. This remains true even if I wander well outside the scope of the input pictures, e.g. different art styles, or Shamiko in a spacesuit wandering Mars, or...
How about Shamiko on a highly improbable forest run? A completely different art style / "medium", in a completely different pose from any of the input images. Mind you, it's lower quality; that's mostly because I didn't do any cleanup or img2img work.
It's still not as good as the real thing. She's missing her tail in both of these pictures — though she had it in others — but even if I filtered for that, it's missing the expressivity it usually has. Shamiko's an emotional girl, and a lot of that shows in tail behaviour...
I doubt I need to tell you that there is no way to say "Jealous Shamiko, with tail curled protectively around Momo". Not yet, at any rate.
That definitely is impressive, and thanks for pointing out it still misses key features! I will say though that those anime-styles, for lack of a better term, seem to do very very well when rendered by an AI. Other less simplified or less flat styles it struggles with much much more (not to say this style is lesser in any way, however).
So this seems to be more of a incremental improvement, from a layman's view. I'm sure it's technologically impressive, but it looks like it won't be generating ref sheets. Thank you for testing!
>Jealous Shamiko, with tail curled protectively around Momo
Why can't this be done? It also seems like an object-composition problem, so assuming the model has some concept of "Momo" like it does for "forest" it would seem to be possible? Is this just a limitation of the Dreambooth finetuning process?
Also if you happen to have more samples of the Dreambooth output, could you share? (I want to see that Shamiko in a spacesuit wandering mars...)
I'm interested to know how well diffusion models can generalize across style (I guess this could be tested by keeping a fixed prompt and varying the initial seed state). Have diffusion models successfully learned some latent space for "style"? This doesn't matter for real-world objects since all apples look pretty much the same, but it matters a lot for 2D art where artists usually have a unique style (which I guess would be defined by proportions, palette, etc.).
It is an object-composition problem, and the AI _can’t usually handle those_.
The tail is also highly stylised, frequently taking up completely inorganic poses such as “jagged pikachu style shock/surprise line”.
One or the other of those might be manageable, at least by generating fifty pictures and picking the best. Both in combination means human input is absolutely necessary. Which doesn’t make the AI useless, by any means; it’s fully capable of acting as a collaborator.
> but the act of training a neural network on datasets like this has already been decided to be legal
?
Has it?
Look, you can argue about whether it’s morally right or not to use models that are fine tuned explicitly to copy the style of someone else, trained on their art without their consent, to make a model that can generate images very similar to the training images.
You can argue about technically of that’s copying, or if lossy compression is copying.
…but legal and moral are different things, and right now, as far as I’m aware:
- it’s only legal because there are no laws specifically making it illegal currently.
- there are active (eg. Copilot) cases challenging this to set a precedent.
- it’s sufficiently ambiguous having a model that anyone can type “a naked picture of a 12 year old” in and get exactly that as output, that stability has nerfed the most recent mode release they’ve done.
- there is a reasonably obvious similarity to other fields where a thing is itself not illegal or bad, but it can enable people to do illegal or bad things, and therefore access, ownership and usage of said things (eg. Hand guns) is heavily legislated.
I think “this has already been decided to be legal” is a blatantly false assumption.
It’s factual in much the same way the average person breaks multiple laws as written every day.
Much of this is simple ignorance and many thing aren’t particularly relevant. 14 states still had sodomy laws in 2003 when the Supreme Court reversed its stance and declared them unconstitutional. At this point there are hundreds of years of crap at the federal, state, and local level much of which changes based on where you happen to be.
What percentage of the US laws have you actually read?
I see two ways of interpreting "companies break the law constantly".
One, that there is always a company out there somewhere acting criminally. If that's the intended meaning, it is a factual statement, but doesn't not carry the original implication that companies do not try to avoid breaking the law. For instance, the fact that there is always a human out their somewhere committing crime doesn't not mean most humans do not actively avoid such.
Two, that any given company breaks the law often. This carries the implication that companies do not worry about breaking the law, but is also factually incorrect.
Legal departments are employed to tell you about legal risks, not to predict the future. Your corporate lawyer isn’t going to go “yeah that’s illegal but it’s totally awesome bro”. The execs might but not the lawyer.
In this case, the riskiest issue isn’t copyright, it’s that it can generate NSFW/CSAM, which governments and payment processors both get really upset about.
It’s not about predicting the future. The point of communicating legal risks is to make assessments about which risks might be worth taking because complying with every single law is impractical.
It’s part of the basic structures where the legal team has an advisory role.
I wouldn’t be so quick to say it’s legal because google image search is legal. Image search is legal because “thumbnails” and “image search” were decided by the court to be sufficiently transformative and in a different marketplace from “images.”
I could very easily see it being argued that “computer generated images” are in the exact same marketplace as “images” so already that case wouldn’t apply and new legal reasoning would be needed.
Which makes it legal. When the courts interpret law it addresses a case and informs going forward. I don't believe, for instance, people can be retroactively tried for abortions during the period Roe V. Wade stood.
I'm not sure this is true; it addresses a case, which means the interpretation already is ex-post-facto. I don't know the answer, but it wouldn't surprise me if, in states with anti-abortion laws standing for which the statute of limitations has not expired, abortion providers could be held legally liable.
Ex post facto laws are forbidden by the US Constitution, both at federal and state levels. [1]
At least, at this time, with current interpretation- "the Supreme Court has explained that people must have notice of the possible criminal penalties for their actions at the time they act" [2] (See also Weaver v Graham[3])
The laws are not ex post facto. They have been on the books, on some cases for over 100 years. States were not enforcing them, because they believed they were constitutionally prohibited from doing so.
Roe v. Wade is a ruling that people have a right to abortions. Removing it removes the right. It doesn’t retroactively remove the right.
A new law that is passed doesn’t have retroactive power
A court ruling that the thing you’ve been doing for a year has been breaking an existing law can absolutely punish you for it. But it’s unlikely to punish random individuals doing things on a non noteworthy scale when a clear understanding of the law in a new context has not been established.
Pedandtly, this is not true. You might think something you’re doing is legal, but when someone takes you to court, it’s because they’re arguing that what you’re doing is already illegal. And if the court sided with them, then what you were doing was illegal the whole time. Courts don’t make laws, they interpret existing laws. Everyone can interpret laws in their own favor, but it doesn’t make things ‘legal’ in the definitive sense
> This is case with anything…anything not illegal is legal, at least in the US…
I think you're dancing around with words here. Let me be super specific:
There's no law specifically saying I can't use a jelly coated toasted to beat someone to death... but there are existing laws that cover 'beating someone to death'.
If you beat someone to death with a jelly coated toaster, you might argue for some obscure reason, your actions are not covered by the existing legal framework around beating people to death, but mostly likely you will not be protected by the claim you 'didn't know it was illegal' to do that.
...because the courts have not ruled that beating people to death with jelly coated toasters is legal.
ie.
a) There is no specific legal precedent or law around something but other laws related to it exist
and
b) Something has been determined to be legal by some precedent / law / whatever
Are not equivalent.
Regardless of what people want to believe, or have opinions one way or another, the assertion, made in the original article, that (b) was true, is not true.
Only (a) is true, and that is, legally, a much weaker statement.
> Section 23 (Adaptations and rearrangements): "(1) Adaptations or other rearrangements of a work, especially a melody, may only be published or used with the consent of the author. If the newly created work is at a sufficient distance from the work used, it does not constitute an adaptation or rearrangement within the meaning of sentence 1."
This amounts to the same as what TFA says about derivative works in general. If it's too close to the original, it may be infringing copyright. Gauging the boundary between "too close" and "not too close" is the bread and butter of copyright courts.
> Section 44b (Text and data mining): "(1) Text and data mining is the automated analysis of one or more digital or digitized works in order to obtain information, in particular about patterns, trends and correlations.
(2) Duplications of legally accessible works for text and data mining are permitted. The copies are to be deleted when they are no longer required for text and data mining.
(3) Uses according to paragraph 2 sentence 1 are only permitted if the right holder has not reserved them. A reservation of use for works accessible online is only effective if it is in machine-readable form."
Again, IANAL, but in my opinion this covers neural networks. "Obtaining information about patterns, trends and correlations" is as close to a literal description of the purpose and function of artificial neural networks as you're going to get in legalese. Paragraph 2 just means that you have to delete your data-mined stash if you ever deem your model complete (but it should not be too hard to argue that ML training is always an ongoing process and thus the mined data will always be required). Regarding Paragraph 3, if I'm not mistaken, the "machine-readable form" part of paragraph 3 is very very very specifically aimed at robots.txt. That is, if you allow your art to appear in search results (which most artists want), then it's also fair game for data mining.
Once again, IANAL, and this is only German law. But I think it's very easy to make a case here that this existing code of law covers AI training as well, as long as measures are taken to ensure that source images cannot be reproduced with any sort of fidelity (and obviously that's a big "if").
Yeah, we'll see how the courts come down on this one. But If we follow exactly the same process as you describe using a human instead of a computer it seems like that would be fine. Artists copy each other all the time.
Also, this looks to me like a situation where generating art is suddenly so cheap and easy it doesn't matter what the courts decide. People can and will ignore the law, because it is trivial to generate new pictures and it is cost ineffective to enforce any restrictions. We're going to see this tech take off. Who knew that art would be the next thing software took out?
This is basically my take - I'm mostly confused by the people up in arms about this (except that it likely makes their work more of a commodity, so I get that fear).
People look at art and make art in that style all the time, now a machine exists that can do that. Why is that unethical? Because they didn't consent for the machine to look at it/learn from it, but they did for humans? I don't think this argument will be able to hold the wave of change that's coming from this new capability. They'd be better off long term learning how to use it.
Nobody creates purely original things in a vacuum, machines won't either.
Automation is okay if it is about taxi drivers or warehouse workers. But when it targets artists some people turn Luddite.
My brother is an artist by the way who does a lot of work with AI, 3D printing and internet. Art will always survive and adapt.
If memory serves it took decades before photography was socially accepted among the art world.
The question now is what value can (human) artists bring besides merely producing images of a certain subject in a certain style. Software has clearly just solved that problem, although the buildup was the last 10-20 years (cnns, gans, style transfer, and now generative language-based models).
But when I think of the value and interestingness of art, there's a lot of intention and meaning in choosing what to draw, the form, etc.
Even from a first glance, the one by David is clearly superior. The scene is so much more striking and interesting to look at. You can see the emotions of the characters, and overall the composition underscores the significance of the philosopher's death, especially in the context of the enlightenment/romantic period. (This is my opinion as a pleb; not an art expert.) But what I do understand, as a computer person, is that these concepts are still beyond what a model can encode in an image as of today.
So although humans are no longer superior in the mechanics of producing images, I think in the higher-level/psychological aspects of "art", there's room for humans, at least for now.
Want to add, I understand the debate is also around the revenue loss in the commission/fanart/online art scene. But from having looked at lots of these over the years, I'd still argue the same thing: that IMO the really good series and artists are good because of their ideas and themes, and not their technique. But if a particular artist's revenue is 90% from drawing lewd fanart, unfortunately it seems like they'll have to adapt and compete, using the skills that humans are still dominant in.
And although I'm just an dumb anon on the internet spewing these ideas, I know I sound harsh, but I think I'm correct. Because the reality is that now, everybody's downloaded the SDv1.4 weights onto their hard drives, and the cat's out of the bag permanently.
I have never used any of those services before. I have never considered using one of the services before because it was always outside my price range.
Now that I have tried stable diffusion and have photo bashed and rendered some concept art for each of the main characters in my novel, I now want to commission an artist to create the 30 or so needed training images so I can ask stable diffusion to spit out my own character in various poses and expressions.
At minimum, that will require a human to render a model sheet of the character from front and back and side and 3/4 and above and below, as well as the emotions on the basic emotions wheel.
If I want the character to be able to wear different outfits, then I will also need to pay for renderings of that character wearing that clothing, all in service of trying to train stable diffusion to be able to remix that character into future images.
Let's also say that I do not have a killer graphics card to be able to train images into a model, luckily, for another $100 fee, the artist will use their existing graphics card to spit out an embedding or a hypernetwork or a VAE or whatever it is that you can use to add custom training to a model and send me that as well as the original set of input photos.
After all of that, I can generate the photos I want...but I will then have to slightly tweak each image so that it has human authorship, even if it's just removing noise and fixing the cursed loops that happen on limbs at times.
In short, I am considering something that is at least $200 for the crappiest cheapest artist out there, multiply that by my six or so main characters, and that is money that I am genuinely considering spending that I would not have even dreamed of entertaining for a moment.
The proof is in the pudding as to whether this thought process will be happening for other stable diffusion users who are able to get images they like, but do not have good rendering skills on their own, weather they too are willing to pay for this or not.
If so, there will instantly become a new type of artist job available, that of the AI art trainer artist.
At the very least, there will be AI art cleanup artists that remove the noise and so-called cursed elements of ai art when used in the concept art stage.
It’s really not that simple. If an artist picks 12 unusual colors to paint a sunset and you copy those exact same colors to also paint a sunset in his style then no that’s not ok.
People on HN have very strong options about this stuff without looking at any of the relevant case law.
I have no idea if that is legal or not, but it is an extremely common practice. For example, that is the scenario when someone on Deviant Art creates fan art of any cartoon.
And why should that be a problem? This is a King Canute and the tide scenario. We may as well bow to reality and admit that it is ok.
Now it’s so easy to share music, it’s meaningless to try to stop people doing it.
Apply 20 years of law cases and punishment for random people and now…
…everyone pays to stream their music.
Right? Wrong? Eh.
I’m just saying, you are kidding yourself if you think that the Powers That Be will just let people decide copyright isnt a thing anymore because of (insert reason here).
Once there is money involved, there will be court cases, and you know, I’ll be shocked if a combination of “needs bigger GPUs to run” and “legal issues” don’t cause these sorts of models to be locked away behind cloud APIs in the future.
It is what it is. Enjoy it while you can; some things are quite predictable, and:
“Law takes a while to catch up with new technology, but it eventually does, and when it does it favours the status quo”
The models are already locked away behind cloud APIs. Stable Diffusion wasn't supposed to happen; OpenAI thought that nobody else could afford to train a U-Net on CLIP at their scale and give it away for free.
I will point out that the usual copyright maximalists have been pretty silent on the issue of AI art. The biggest opposition to AI is coming from the Free Software community - i.e. the people who want to abolish artists' ownership over their work outright.
The Free Software movement originates in academia and has academic value - they don't care about receiving direct compensation, but attribution is critical, and attribution is what image generation models can't provide.
I think your example doesn't quite work here. People pay for music now because handing Spotify or whoever ten dollars a month is a easier then torrenting, easier then managing directories of mp3 files and moving them to your phone, and has value adds in the form of discovery.
You don't need to pay to listen to music if you don't want to. The corpus of good music on YouTube for free is probably bigger than what you can listen to in a lifetime. If it isn't already it will be in time.
Napster's model won that war. Effortlessly. If you're paying for your music, you are paying on your terms based on the value you think is being provided to you. It isn't a legal framework making you do it.
The difference between music and AI art is that music is still made by humans. The music industry currently has a monopoly on producing new music, so of course they have leverage to get paid for that new music.
My point is that Napster-to-SD is not a fair comparison. Napster didn't eliminate or replace human artists, while SD certainly did. Therefore, it's not fair to assume the government can regulate themselves out of this one like they did with Napster. Because even though Napster enabled widespread piracy, the human musicians still had leverage in that they were needed to create new music.
So my answer to the question posed by parent comment of "will SD play out the same way that Napster did?" is "no" because there are fundamentally different economics at play.
Obviously, if a good generative model comes out, the music industry will be in a similar boat to the art industry right now. Google was working on it in 2017 (https://magenta.tensorflow.org/performance-rnn) but I don't know if they've made any progress since.
Spotify won because it's better than The Pirate Bay. The Playlist on Netflix explains why Spotify surpassed TBP and how the record labels still had to surrender even though they "beat" the pirates.
IAAL. Regardless of how the Copilot case develops at the trial court level, anyone who thinks the Roberts' Supreme Court is ultimately going to kneecap this technology needs to reboot.
YEah, I think people also need to give up on saying "well precedence says" with the current SCOTUS roster. They don't actually care about precedent when it doesn't jive with their agenda. It's hard to believe that the mightest lawyers in the land think precedent is unimportant but we've already seen it a few times and this SCOTUS is just getting started.
Yes. When artists say things like "this really feels like it should not be fair use" we are dismissed with "well it's legal!". Changing what "fair use" covers is a possibility.
Unfortunately art is a hard business to make a lot of money in and the vast majority of the people unhappy about this do not have the money to mount any kind of legal challenge to the corporations developing these databases, so we are basically fucked.
Which is in some ways nothing new, the average Internet user has no concept of creator's rights and will blissfully assume that if something's not behind a paywall, it's free for any use. It feels even shittier and worse when this is being done on an industrial scale by these AIs, though.
Even if artists could sue this is assuming there is someone making enough money to sue.
Open source models will likely be dominant (this was how Stable Diffusion got popular, it's not clear their new neutered 'safe' model will be as popular or a successful business).
This is the new reality even if the bigco gets sued out of business. Artists need to accept that reality eventually. Just like all "panics".
Fortunately AI generators are not a direct replacement for artists, I've seen it used as part of the design process, but that still takes talent. Maybe it will get better but it's not a one-stop shop for business use-cases. More likely it will be AI+artist, not AI replacing artist.
If an open-source model gets DMCA'd, then getting a copy becomes a hassle. This does not entirely stop things - people still have massive collections of Nintendo ROMs and downloaded films, but it's work to get it running in a way that, say the Stable Diffusion tool that Clip Studio Paint just announced is not.
A real open-sourced model trained on images that were explicitly licensed for such uses would be interesting. It would cost money but it would also not be hiding a ton of its actual cost by training it on images whose creator never imagined that "being dumped into a vast dataset" would be a thing that would happen, as well as images that were fine in their original context as "fair use" but got sucked in by the web crawlers feeding the dataset. You want to train an image-generating machine on my work? Ask my fucking permission and if I say "pay me" then you either negotiate a price we can both live with, or you do not put it in your dataset.
I do not accept your reality. Sue 'em all until you've gotta train your own damn datasets, or buy ones that have worked out the proper licensing. If Photoshop can be made to refuse to scan currency, then these things can be made to be fair to the artists whose shoulders they rest upon.
AFAIK, from the discussion here, training the network has been established as fair use on the US. But that doesn't apply to use the network's results in any way.
As far as I know that hasn’t been established yet but is presumably what would be decided (especially in an academic environment). But yes the output is a totally different question.
Although not being explicitly illegal does make something legal, the phrasing "has been decided to be legal" suggests a debate, a conscious decision, and perhaps an overturned law. That is not the case as far as I know. It is legal (because that is the default), but no decision has been made.
There is a lot of case law on what a derivative work is. A model that outputs an identically image to the training doesn’t mean the trained model is infringing or a derivative work.
> ... it’s only legal because there are no laws specifically making it illegal currently.
One might argue that any unlicensed use of copyrighted material is misuse (unless the courts have ruled it is fair use.)
Only the copyright holder may bring action against a potential misuse. Typically, individuals tend not to have deep enough pockets to hire the lawyers to bring the action. And during discovery it may be found that the model was trained on works-for-hire made by humans replicating the original style. Or appeals may continue on at more legal cost, without guarantee of recompense.
Most will decide it’s just not worth it. Until a company similar Getty buys up all the copyrighted material and, in addition to sensible suits, brings spurious legal action against individuals and turns the tables.
In a lot of ways, whether it has or not is moot in the medium-term.
Court cases / legal clarification may outlaw Stable Diffusion or DALL-E for having been trained without the consent of the original artist on their copyrighted artworks. But if this technology proves valuable and viable, the likes of Disney, WPP, and Omnicom Group will pay a few dozen artists to create enough work to seed an engine that can generate 50 million wholly-owned lookalikes.
Copyright protection will ultimately shield the megacorps from competition by the common folk, not artists from competition by corporations.
> it’s only legal because there are no laws specifically making it illegal currently.
No, it's only legal because nobody has litigated a case all the way to the Supreme Court. We all thought that APIs weren't copyrightable, and then, suddenly they were (or worse, were "assumed to be" but not explicitly ruled upon).
In the EU, training a model on copyrighted data does not infringe copyright. This is explicit, codified law as of the latest EU Copyright Directive[0]. In the US, AI researchers are more or less hoping that Authors Guild v. Google forms controlling precedent against lawsuits against the AI companies. While this is not done-and-dusted case law, I can see the logic. The model is capable of, and is intended to be capable of, creating novel art.
In neither case is it legal to use outputs of an AI that infringe an already-existing copyrighted work. This means anyone using it to generate production-ready artwork is exposing themselves to potential legal liability if their AI winds up regurgitating training set data. This is what I worry about way more than just "are the model weights infringing copyright".
As for morality:
- Absolutely none of the current generative art systems are trained from scratch on ethically-sourced datasets. The AI companies just assume that because they need ungodly amounts of training set data, that they are morally entitled to get it, because they spend more time staring into the eyes of a basilisk[1] than worrying about the world they already live in.
- Multiple companies getting into generative art have gone above and beyond in giving human artists the middle finger. DeviantArt decided to blow all their good-will from defending against NFT nonsense by making their AI training program opt-out, which is NOT HOW CONSENT WORKS. Mimic[2] and Dreambooth are basically artistic impersonation tools that have specific moral implications beyond the general practice of AI art.
Whether or not this becomes actual law is... well, I'll put it to you this way. Stability AI's Dance Diffusion is actually trained on an ethically-sourced, public-domain dataset. Why? Because they're afraid of being sued by the RIAA. Despite the name, copyright maximalists are less about maximizing the rights of artists and more about building moats around large publishers. And generative art does not threaten[3] those publishers, so it will not be banned.
[0] Yes, the same one that mandated upload filters on video sites.
[1] Roko's Basilisk posits the idea of a superintelligent AI - one that can eat everyone's brains and emulate them perfectly - constructing a perfect hell for people who didn't build it as a way to threaten those people in the past to build it.
Longtermists can burn in computer-generated hell. Time discounts exist for a reason.
What I find weird is that humans do the same and no one mentions that; I have 2 professional (they have lived of it for decades) artist friends; one makes Giger art (paintbrush, exactly the same style but no copies, original but if you see it you are going to say Giger) and another one Vermeer, same thing. There is no discussion if that’s moral or legal, what’s the difference? The Giger one has bought a farm of GPUs a few months ago and is adding training to models for animations. He loves it.
Same with copilot; people copy shit from GitHub and SO all the time without mentioning copyrights and a lot of code people cough up is just ‘stolen’ from someone without remembering who/where it was; what’s the difference?
I'd say there's a good chance both of those artists are going to grow "beyond" the artist they are emulating (and by "beyond" here I don't mean better than but rather grow into something unique).
Lots of bands (nearly all?) learn their craft by playing covers. At some point the artists in the group start to find their voice.
>For most hobbies/professions, there are barriers to entry. For some it’s expensive equipment, for others it’s training and hard work to acquire a skill.
Barriers to entry? Training and hard work are barriers to entry?
So the next time when you go to the dentist, tell him that you can fix your tooth with a text prompt.
> Stable Diffusion? Well, that’s free-ish. Problem is, you’ll probably want to run it locally, which requires a really, really beefy graphics card. I was struggling to run it on a Vega56 - a GPU that goes for ~$150 used now - so I went out and got a RTX3090 for about $1,000. If you’re already a gamer with a GPU with 8Gb+ of VRAM you’re probably good, but for most people this is a bit absurd.
I agree with the problem, on platforms like AWS you'll even need to send a manual request so they would let you use instances which can run SD. On the other hand there's already something like replicate.com, which allows you to run the SD like an API. I hope there will be more services like this.
A couple of years ago I wrote the script for a graphic novel. I would have liked to illustrate it myself (I went to art school after all), but I was too busy running a startup at the time, so I pitched it as a collaboration to about 30 different artists that I thought could do a great job with it. I could not get a single person to bite.
In my pitch I proposed generous royalties and cash compensation to the artists. All of them essentially said that they weren't taking commissions. I'm sure this is in part due to the fact that I have never been involved in the graphic novel world, so they'd be taking a chance on someone new. Still, it seemed there was no amount I could pay to get someone to work with me.
Now I am reviving this graphic novel idea and still looking for someone I can pay to help me work on the project. However, the more synthography advances as a technology the more it seems like I should explore it as a path for this project. It would help me develop the book much faster than I can on my own. At minimum it could help me get the story board in place.
Maybe some day synthography can help artists scale up their output, in the same way Michelangelo developed a workshop of apprentices. I would be more than happy to pay an artist's AI apprentice if I can't work directly with the human.
Hi, I run http://synapticpaint.com/ and using AI image generation for graphic novels/comics is one of the directions I'm exploring. If you're interested in collaborating to make your graphic novel a reality (I'll provide the tooling in return for product feedback), please email me at the email address in my profile! (I poked around on your site but couldn't find an email address.) Thanks!
I've had similar experiences trying to commission works. Turns out artists are difficult to work with almost by definition. And the few who actually act like they're providing a service you've agreed to pay money for are few and far between.
If you don't mind answering, I'm really curious how much cash you were offering up front and what the page count you were commissioning was. Been talking to some art consultancies recently and I'm curious to how your experience compared to mine.
I think you're wildly underestimating the amount of effort and very specific skills that go into making comics, it's either a labor of immense love for the art form, or getting paid a large amount of money up front to work with an established writer, nobody is looking for an idea guy
It seems there are a few distinct things we have to consider:
How is art defined? How is it defined by artists and how is it defined by consumers. Some artists may want to define it such that it's made by people. Much fewer consumers would make that a requirement (aside from some art collectors). Consumers will care much less of how some art was generated and they will dictate demand.
I found this funny as well, but on reflection I think the author is talking about the specific effect of changing the seed but not the prompt. To me it highlights a weakness of AI imagery — no intent, no progression, no story.
Eh, yet. I can see where the prompt just gets extended to, "in a hilarious series events in the style of buster keaton" and all of a sudden your vampire-toothed anime furry in cyberpunk clothes is flying down the side of a building, being saved at the very end by a sunshade they flop into. But I don't know if I'm going to live long enough where AI makes new Buster Keaton shorts using commonplace elements around them in new and exciting ways. Like would AI know that you can hang from the hands of a clock, which move over the course of a day? to me that's some creativity that was years ahead of its time.
Now that I've attempted to predict the far-from-now future, I'm sure we'll see this in 6 months.
If artists can't adapt they'll just be left behind - not much value to be lost if they could be replaced with Stable Diffusion. Low hanging fruit, too bad their etsy store won't bring in $8k a year anymore
Artists provide different products from image generation models, especially artists making physical objects, so they probably won’t be replaced at all.
good guide, I've been working on stable diffusion a lot as well and ran into many of the same problems with extra body parts and have developed a few techniques that remove a lot of the extra body parts, but still refining it. He is also right about stable diffusion not being free as it is a PITA to get the GPUs running and software stack setup. I'm using the body cleanup techniques and other algorithms to improve output at https://88stacks.com which is a free stable diffusion custom model generator api. its running on my research cluster to train models. Please check it out and give me feedback :)
I can understand the author's feelings on this. It's fun, and it's exciting that with no knowledge of a domain the author can get interesting results. It can "feel like" creativity, even though literally anyone can get good results.
Similarly, AI-remixed¹ music is getting better and better. I want to know how the author will feel when their music² is used to build a model that allows anyone to create infinite music in their style and beyond, much of which will be enjoyed far more widely (because it will be free or royalty-free) than the source material ever will.
¹ It doesn't seem fair to call it "generated" or "created" when it'd be useless without an enormous corpus of source material powering the thing.
I'd be fine with it being used to train models. The music world is already extremely derivative and trendy. In western music anyway, we have 12 notes to go around and often only use a subset per song. Additionally, much of a song's structure is formulaic, even with music as experimental as my own. I've used AI tools for composition (though I've always had to modify their outputs), pretty much all musicians use samples, etc. If anything, I see the use of AI as a step towards having less reuse in music because of the ability for generes to bleed together.
> Right now, your music requires attribution and can't be used for commercial purposes.
Sure, but fair use still exists. If someone takes just a single drum hit, reverses it, and coats it in distortion and reverb, but some how I find out they did it, I can sue them. But I can sue over anything, that doesn't mean I'd win. I'm pretty confident they would, and would agree they should because it's fair use. I think training an AI model is fair use as well.
> Should visual artists have the same ability to choose the rights they reserve?
Yes, and within the bounds of fair use as musicians that I've stated above. If I photoshop out a single eye, let's call it 50x100 pixels of a 5000x5000 image, and use it to make a neat mash up art, I think that is transformative enough to be fair use. Similarly, I think the AI model is transformative enough.
In either case, there are limits. If someone makes a model fine tuned on my specific music, that's wrong. Similarly, if someone makes a model fine tuned on an artists specific work without their permission, that's wrong. Using an artist name (music or drawing) in a prompt is gray, but I personally think leans on being acceptable, especially if that artist is dead.
I'm impressed. this is the first critique I've seen that actually engages with the topic in a fair way. As of now, there's been just way too much noise to get to the detail of what exactly is wrong with AI art.
As for me, I just feel that it should be looked at the same way that photography is looked at; that it's a new medium all to itself. You wouldn't enter a photograph in an oil painting competition, so you shouldn't enter AI art into a drawing/sketch art competition either. It would also be useful if there was an acknowledgement that AI was used in some elements of the art, but like as not there will always need to be a human agent that makes corrections in what is created.
Maybe visual art will respond to this like it did after the camera came out. Suddenly artists were free to interpret the world through their own lenses.
I guess the difference that I see is that with photography I’m not generating a completely random image every time I push the shutter button. I think the corollary might be if I pointed my camera at some trees and pushed the shutter button resulting in my camera then picking a random picture that someone else has taken of trees anywhere in the world and I now have to hit the button many times to find the “right” one.
I understand that photography generates waste images. I just don’t feel like it’s the same.
Personally, I don't consider myself an artist and don't believe I have an understanding of what it means to be one, which will bias my thoughts on the subject. Still, I'm having some reservations about the ability to endlessly churn out AI-generated content.
In my view, in the past it was believed that to be an artist and produce the output you admire, you had to learn things. You had to be patient, and spend thousands of hours learning a skill. But I don't think it starts and ends with just 10,000 hours of brushstrokes. All that time and effort likely causes other neurological changes owing to the situations that being a traditional artist forced you into, like being exposed to other people and communicating with an audience.
The ability to generate art with prompts bypasses all that time and effort. Using Stable Diffusion, I have made incredible pictures (that I don't consider art), but at the end of the day I still feel like just another person chasing yet another rush of dopamine, the same as it's always been. It's an enabling device. I have never been able to summon the will to practice drawing or painting for enough time to see results, and that hasn't changed. The only difference is that I have the end results, which is mostly what I wanted in the first place, and it's pretty much all I'll ever get with this setup.
But it is often repeated that "focusing on the end results" when trying to learn a creative skill results in bad outcomes. It
is a character flaw that someone has to overcome to get better. Now I feel that 70% of the road to "the end result" can be bypassed without the need to work towards that additional knowledge and virtue.
When artists speak of AI damaging the integrity of their profession, I think this is part of what they mean. For the first time in human history, it is now possible to be a better "artist" without growing into a better person, or developing a wider-scale notion of what existence or the world outside is. The folk assumption that growing up as an artist changes you in fundamental ways (by forcing you to study the craft and observe the world for what it is) has been violated.
The people who treat art as a profession or hobby to work at over thousands of hours, for example, will probably share a lot of significant interactions and bondmaking as their skill gradually improves over the years; not so with a person that can generate pseudo-masterpieces within minutes.
I'm going to guess that the long-term downsides of AI art are going to be baked into cultural norms instead of the limits of silicon. There has already been significant backlash from art circles when tech-oriented people outside of the scene train models in the style of people like Kim Jung Gi, who had just recently passed away. Many people's rules about "art" have gone unwritten for so long because this current situation was unthinkable only a decade ago. Regardless of intent, being seen as plagiarizing the style of someone who is no longer alive isn't a good look.
Still, what I always want to keep buried in my subconscious about how AI art is used and treated is that my insatiable interest in the subject is fueled by my lack of (ability/drive/bravery) to express myself through my own actions, whereas my embarrassment is spared by delegating to a machine to do the work for me and guess what the inner workings of my imagination look like. If this is something I need to overcome, it's only going to get harder for me to do so as the state of the field advances further.
The word that keeps coming to mind when I look over the things I generated is "intent". It doesn't feel like I "made" the things I did, but that I "found" them. There's not a lot of insight to be gleaned except the keywords that each subset of popular culture finds the most useful or interesting in regards to prompt engineering during a particular timeframe. My preo...
OP here. This is phrased beautifully and at least partially reflects my own views. I especially like how you said "It doesn't feel like I "made" the things I did, but that I "found" them".
This is a big part in why I enjoy vastly different kinds of music creation. When I work with a traditional DAW and build a track from scratch, I'm sculpting and crafting an artwork. When I work with modular synthesizers and generative compositional tools (AI or not) I feel like I'm navigating sonic landscapes to find interesting destinations.
AI art is the latter, and while it's fun, it's a different feeling during creation all together. In music, I can typically hear which the artist was doing in a given song. With Stable Diffusion and DallE, I don't know that I can as reliably. This is why the divide is so interesting to me. We have to radically different avenues, each with their own merits in how fulfilling they can be, to choose from which produce similar results.
I will admit, there is something to the reward of making something hand crafted that at least to the artist, adds value, even if that value isn't (typically) transferable to the end user.
Songwriters have been using rhyming dictionaries since the day they were first put on sale. It’s a similar kind of process of discovery. You still have agency and individual inspiration.
Like, this isn’t some kind of hack’s approach to art. Literally every working lyricist that I know uses a rhyming dictionary from time to time. Why? It’s fucking work! There are deadlines!
With regards to AI visual art… the best results I’ve seen have been from people with strong visual art skills. Aesthetic choices still matter a lot when you’re tweaking the prompt, model, hyperparameters, etc.
273 comments
[ 2.2 ms ] story [ 276 ms ] thread> txt2img prompt crafting is a bit of an art in its own right, as getting the model to spit out what you want isn’t always trivial.
I don't find the output images from a lot of "AI" art to be artistically interesting because an "AI" made them. The output might be novel, or even kind of surprising, but still just the output of code. Someone might say the code of an "AI" art machine could qualify as art. I often find the txt2img prompts artistic in a way that the output images lack, because they are representative of human imagination, often like surreal poetry, or like musique concrete, cutting and pasting segments around on a tape. This is not to deny that the output of "AI" art is impressive. It is, frankly. But as a matter of machine, computational, output rather than human artistic merit.
It comes down to a simple thing: if you showed me two images and you told me a human made one, and "AI" made the other, I will always find the former artistic in a way that I would never find the latter.
This is also not to touch on the licensing issues that arise from these tools. That's its own knot to untangle.
Whatever line you draw will leave several canonical figures on the wrong side of it. Maybe that's OK and you'll draw your line in the sand. Just don't pretend it's a non-controversial consensus.
I find "what is art" discussions impossible to resolve and untangle from prejudices and biases. The least problematic answer I've found comes from John Carey, paraphrasing: "art is whatever someone has decided it to be". In other words, if I decide something is art, then it is.
The problem shifts slightly to a more interesting way to pose this question: why some art has more value than some other art (or even "does some art has more value than some other art")? Equally difficult to resolve but more prone to highlight prejudices and biases mentioned above.
I'm curious about the possibility of the commissioner role for this situation.
From my point of view, the AI is the artist and prompting an AI to produce a specific image is akin to commissioning an artwork.
I think we have moved beyond human agency, and creation of art is reduced to the simpler constituents, the roles of artist and (if there is one) commissioner. The request to make an artpiece can also come from a machine.
Also, we're comparing an art 150 years in the making (with its schools, philosophies, heroes) with one in its infancy.
Oh, for sure. As for the rest, well I might've also missed the whole writing prompt thing. But I've always felt as if reverse engineering the thing.
But does that make it more artistic than something generated by AI which was coaxed out by a human applying their expertise?
Would you say the same of general intelligence? IE, general intelligence (artificial or otherwise) requires an agent?
It doesn't matter anymore that you emotionally value, or want to emotionally value, human-made art more than AI-made art, if you can't distinguish the two except based on other people's representations of how it was made.
Unless you watched a (non-augmented) human make a piece of art, you have no assurance that a human didn't touch up an AI generated art piece or simply curate a collection of fully AI-generated art.
ETA: I'm not trying to put down recognizable human-created art that's already been created, especially historically notable works. I'm not sure what all those replies are trying to get at. Of course there can continue to be an art-collecting market for human-made works made before the AI-art era.
The same holds for AI art.
People will pay more for the original because it's rare.
If an artist who studied Van Gogh intensely made an original piece in Van Gogh's style, does it have equal artistic merit as one of Van Gogh's originals?
AI is just going to be another aspect of provenance, and if it's found out an art piece had AI used to make it, it won't be forgotten or ignored.
I find it similar to watching chess. AI/Chess engines will always be better than humans, but it's exciting to watch a human play to see what they can do.
For example, a woolen blanket made by my mother is much, much dearer to me than an equivalent woolen blanket made in a factory.
I see it very likely that certified-provenance human-art may command high prices in the future; but also for physical part, it's pretty trivial and doesn't have to be so posh and fancy - anything from street art to art galleries and art collectives/workshops.
The problem isn't for existing artists (except in terms of ethical issues)—it's for new/budding artists, who will have to contend with challenges to the authenticity of their work. How can they prove that they put their blood and sweat into making a piece of artwork by hand when an AI could've generated something equally passable?
The art is always human-made. Missing attribution doesn't mean "machine made it". Humans developed the AI, ran the AI, created the training data, picked the non-horrible sample from the output, etc.
When I started doing this, I was uncomfortable thinking of myself as the author of the image in any way. I'm okay with it now. There's some skill I am expressing, even if not much, and I am connecting with an audience.
I feel like the majority of "art" we consume falls in the second category.
As a historical note, this argument is over 50 years old. In 1965, an engineer at Bell Labs generated pseudo-random artworks inspired by famous artist Piet Mondrian. He found that a) people couldn't tell if the artwork was generated by Mondrian or the computer, and b) people generally liked the computer's art better.
https://www.historyofinformation.com/detail.php?entryid=4437
To take a selfie, human intervention is minimal: someone decides they want to take a selfie; take dozens of them, pretty much at random; choose one they like; publish it.
Similarities with the stable diffusion process are striking. And yet, could you consider a selfie "artistic"? Are all selfies artistic? What if Annie Leibovitz or Marina Abramovic take a selfie?
What if Marina Abramovic generates a million images of a rabbit with stable diffusion and papers a whole warehouse with them? Is this art?
Edit: syntax.
https://www.npr.org/sections/thetwo-way/2017/09/12/550417823...
But most selfies probably aren't high value art.
If Marina Abramovic were to take a bunch of selfies, or generate a bunch of AI images, and exhibit them, that might be considered high value art. A famous artist can literally tape a banana to a wall and it'd be considered high art:
https://www.vogue.com/article/the-120000-art-basel-banana-ex...
Nit: 2 weeks ago there was an article about getting SD running on an iPhone. The app isn't very time- or battery-efficient, but it just barely works on my (previous-gen) device: https://news.ycombinator.com/item?id=33539192
Still, 2 minutes on my phone. I remember Bryce taking half an hour to render lower resolution images back in the late 90s.
That said, you can just rent a GPU instead of buying it right away. It's not very expensive, and can be cheaper overall depending on your usage. It's also the only sane way to experiment with training as you need a lot of attempts.
This stuff can really devour any hardware you can possibly throw at it.
I agree, this is an underrated option and I'm not sure why it doesn't get more attention. I've been using stable diffusion on a cloud machine, and, say $20 will get you very far - probably thousands of images generated and a few sessions of fine tuning. It's way more affordable that its apparent popularity would suggest
Artists should rejoice, in my opinion, mastering this type of tool could dramatically increase their productivity, and thus their salaries.
However - I do wonder if this will increase the value of non-digital arts and crafts. They can't easily be substituted for - which is probably one of the key factors in establishing value in the art market.
I take it you haven't been keeping up. Stable Diffusion 2 and Midjourney are getting really, really impressive. Probably a 10x increase in quality over the original DALL-E already.
Take a look at Jodorowsky's Tron (Tron in the style of Jodorowsky's Dune): https://www.djfood.org/fantasy-jodorowsky-tron-visualisation...
Everyone who's dismissing this because it imitates the style of another artist (as if that was a hard technical limitation rather than a specific prompt) or lacks the je nais se quoi of "real" art is missing the point. No one would ever hire a concept artist capable of doing this when they can hire someone for a fraction of the salary to write prompts that return acceptable results in a fraction of the time.
Pretty much like power tools for carpenters. The house won't build itself, but the process will be improved.
Do they? Show me something truly novel either in terms of art or message created by Hollywood in the last decade. Art and entertainment are risk averse industries - novelty is risk, genre is safe, and within genre you have well defined aesthetics, themes and material from which to derive new iterations on a theme.
The mashups in the blog posted upthread (particularly the Giger/Henson posts[0]) look far more visually striking to me than most modern sci-fi. I want to see that movie based on the look alone, and the look is literally just "what if Dark Crystal, but xenomorphs?"
I work in the video games "industry", we need concepts and mockups, but a much larger part of the production is assets, meshes, textures, materials, shaders...
We're seeing huge improvements, from photogrammetry to simplified production pipelines and more powerful light models.
ML based tools will also help to iterate faster, but the key point is control, human control.
At first yes, faces and hands weren’t perfect but in the recent models and especially with model blending I can now generate super realistic photos of humans with perfect faces and eyes and perfect hands. Basically what is possible and the quality is rocketing forward right now and I can’t even begin to imagine where it will be in a year.
I would also add, the advances in prompt engineering, in-painting and image to image generation mean getting exactly the result you want with the composition you want is also very possible. If you’ve only generated a few images then you really have no idea of all of the tools and options you have. Over the last few months I’ve probably made 5000-10000 images and my own personal skill level for being able to get what I want has gone up massively.
Salaries do not increase with productivity. Rather, they tend to remain constant or even decrease as productivity rises, as technology makes that level of productivity easier to hire for, and thus less costly for the employer.
How many office jobs have been replaced by computers?
More productive. And I would argue still mostly underpaid given the value they create.
> How many office jobs have been replaced by computers?
Most of them?
This was true maybe a month ago, but DreamBooth makes this possible
I've read people making comics and generate many different scenes say v4 now allows consistent character design.
I took ten high-quality pictures off Pixiv, spent five minutes cropping them, and ran Dreambooth overnight on a 3090 — this turned out to be enormous overkill, the final model was massively overtrained. Intermediate ones were fine.
The output is perfect; even higher quality than what the input model was already good at. This remains true even if I wander well outside the scope of the input pictures, e.g. different art styles, or Shamiko in a spacesuit wandering Mars, or...
https://usercontent.irccloud-cdn.com/file/w300/l9rBbcFo
Trust me when I say that 'magical girl shamiko' is not usually a thing. The technology works.
https://cdn.discordapp.com/attachments/807260261747785759/10...
How about Shamiko on a highly improbable forest run? A completely different art style / "medium", in a completely different pose from any of the input images. Mind you, it's lower quality; that's mostly because I didn't do any cleanup or img2img work.
It's still not as good as the real thing. She's missing her tail in both of these pictures — though she had it in others — but even if I filtered for that, it's missing the expressivity it usually has. Shamiko's an emotional girl, and a lot of that shows in tail behaviour...
I doubt I need to tell you that there is no way to say "Jealous Shamiko, with tail curled protectively around Momo". Not yet, at any rate.
So this seems to be more of a incremental improvement, from a layman's view. I'm sure it's technologically impressive, but it looks like it won't be generating ref sheets. Thank you for testing!
Why can't this be done? It also seems like an object-composition problem, so assuming the model has some concept of "Momo" like it does for "forest" it would seem to be possible? Is this just a limitation of the Dreambooth finetuning process?
Also if you happen to have more samples of the Dreambooth output, could you share? (I want to see that Shamiko in a spacesuit wandering mars...)
I'm interested to know how well diffusion models can generalize across style (I guess this could be tested by keeping a fixed prompt and varying the initial seed state). Have diffusion models successfully learned some latent space for "style"? This doesn't matter for real-world objects since all apples look pretty much the same, but it matters a lot for 2D art where artists usually have a unique style (which I guess would be defined by proportions, palette, etc.).
The tail is also highly stylised, frequently taking up completely inorganic poses such as “jagged pikachu style shock/surprise line”.
One or the other of those might be manageable, at least by generating fifty pictures and picking the best. Both in combination means human input is absolutely necessary. Which doesn’t make the AI useless, by any means; it’s fully capable of acting as a collaborator.
?
Has it?
Look, you can argue about whether it’s morally right or not to use models that are fine tuned explicitly to copy the style of someone else, trained on their art without their consent, to make a model that can generate images very similar to the training images.
You can argue about technically of that’s copying, or if lossy compression is copying.
…but legal and moral are different things, and right now, as far as I’m aware:
- it’s only legal because there are no laws specifically making it illegal currently.
- there are active (eg. Copilot) cases challenging this to set a precedent.
- it’s sufficiently ambiguous having a model that anyone can type “a naked picture of a 12 year old” in and get exactly that as output, that stability has nerfed the most recent mode release they’ve done.
- there is a reasonably obvious similarity to other fields where a thing is itself not illegal or bad, but it can enable people to do illegal or bad things, and therefore access, ownership and usage of said things (eg. Hand guns) is heavily legislated.
I think “this has already been decided to be legal” is a blatantly false assumption.
This is case with anything…anything not illegal is legal, at least in the US…
(It’s legal the same way Google Image Search is.)
This is why we do heath inspections etc, it’s all about shifting the risk vs reward calculation.
That's obvious hyperbole and not a useful, or even factual, rebuttal.
Much of this is simple ignorance and many thing aren’t particularly relevant. 14 states still had sodomy laws in 2003 when the Supreme Court reversed its stance and declared them unconstitutional. At this point there are hundreds of years of crap at the federal, state, and local level much of which changes based on where you happen to be.
What percentage of the US laws have you actually read?
One, that there is always a company out there somewhere acting criminally. If that's the intended meaning, it is a factual statement, but doesn't not carry the original implication that companies do not try to avoid breaking the law. For instance, the fact that there is always a human out their somewhere committing crime doesn't not mean most humans do not actively avoid such.
Two, that any given company breaks the law often. This carries the implication that companies do not worry about breaking the law, but is also factually incorrect.
In this case, the riskiest issue isn’t copyright, it’s that it can generate NSFW/CSAM, which governments and payment processors both get really upset about.
It’s part of the basic structures where the legal team has an advisory role.
I could very easily see it being argued that “computer generated images” are in the exact same marketplace as “images” so already that case wouldn’t apply and new legal reasoning would be needed.
At least, at this time, with current interpretation- "the Supreme Court has explained that people must have notice of the possible criminal penalties for their actions at the time they act" [2] (See also Weaver v Graham[3])
This might be subject to change.
[1]https://constitution.congress.gov/browse/essay/artI-S9-C3-3-...
[2]https://constitution.congress.gov/browse/essay/artI-S9-C3-3-...
[3]http://cdn.loc.gov/service/ll/usrep/usrep450/usrep450024/usr...
A new law that is passed doesn’t have retroactive power
A court ruling that the thing you’ve been doing for a year has been breaking an existing law can absolutely punish you for it. But it’s unlikely to punish random individuals doing things on a non noteworthy scale when a clear understanding of the law in a new context has not been established.
I think you're dancing around with words here. Let me be super specific:
There's no law specifically saying I can't use a jelly coated toasted to beat someone to death... but there are existing laws that cover 'beating someone to death'.
If you beat someone to death with a jelly coated toaster, you might argue for some obscure reason, your actions are not covered by the existing legal framework around beating people to death, but mostly likely you will not be protected by the claim you 'didn't know it was illegal' to do that.
...because the courts have not ruled that beating people to death with jelly coated toasters is legal.
ie.
a) There is no specific legal precedent or law around something but other laws related to it exist
and
b) Something has been determined to be legal by some precedent / law / whatever
Are not equivalent.
Regardless of what people want to believe, or have opinions one way or another, the assertion, made in the original article, that (b) was true, is not true.
Only (a) is true, and that is, legally, a much weaker statement.
I have investigated this a few months ago while researching a radio show segment on AI art. In German law, the relevant statutes appear to be section 23.1 and section 44b of the copyright code. The original text is at https://www.gesetze-im-internet.de/urhg/__23.html https://www.gesetze-im-internet.de/urhg/__44b.html and below is a Google Translate result that I have proof-read.
> Section 23 (Adaptations and rearrangements): "(1) Adaptations or other rearrangements of a work, especially a melody, may only be published or used with the consent of the author. If the newly created work is at a sufficient distance from the work used, it does not constitute an adaptation or rearrangement within the meaning of sentence 1."
This amounts to the same as what TFA says about derivative works in general. If it's too close to the original, it may be infringing copyright. Gauging the boundary between "too close" and "not too close" is the bread and butter of copyright courts.
> Section 44b (Text and data mining): "(1) Text and data mining is the automated analysis of one or more digital or digitized works in order to obtain information, in particular about patterns, trends and correlations. (2) Duplications of legally accessible works for text and data mining are permitted. The copies are to be deleted when they are no longer required for text and data mining. (3) Uses according to paragraph 2 sentence 1 are only permitted if the right holder has not reserved them. A reservation of use for works accessible online is only effective if it is in machine-readable form."
Again, IANAL, but in my opinion this covers neural networks. "Obtaining information about patterns, trends and correlations" is as close to a literal description of the purpose and function of artificial neural networks as you're going to get in legalese. Paragraph 2 just means that you have to delete your data-mined stash if you ever deem your model complete (but it should not be too hard to argue that ML training is always an ongoing process and thus the mined data will always be required). Regarding Paragraph 3, if I'm not mistaken, the "machine-readable form" part of paragraph 3 is very very very specifically aimed at robots.txt. That is, if you allow your art to appear in search results (which most artists want), then it's also fair game for data mining.
Once again, IANAL, and this is only German law. But I think it's very easy to make a case here that this existing code of law covers AI training as well, as long as measures are taken to ensure that source images cannot be reproduced with any sort of fidelity (and obviously that's a big "if").
I had to say it.
Also, this looks to me like a situation where generating art is suddenly so cheap and easy it doesn't matter what the courts decide. People can and will ignore the law, because it is trivial to generate new pictures and it is cost ineffective to enforce any restrictions. We're going to see this tech take off. Who knew that art would be the next thing software took out?
People look at art and make art in that style all the time, now a machine exists that can do that. Why is that unethical? Because they didn't consent for the machine to look at it/learn from it, but they did for humans? I don't think this argument will be able to hold the wave of change that's coming from this new capability. They'd be better off long term learning how to use it.
Nobody creates purely original things in a vacuum, machines won't either.
My brother is an artist by the way who does a lot of work with AI, 3D printing and internet. Art will always survive and adapt. If memory serves it took decades before photography was socially accepted among the art world.
But when I think of the value and interestingness of art, there's a lot of intention and meaning in choosing what to draw, the form, etc.
For example, look at The Death of Socrates and then compare it with the three other paintings of the same scene here: https://en.m.wikipedia.org/wiki/The_Death_of_Socrates.
Even from a first glance, the one by David is clearly superior. The scene is so much more striking and interesting to look at. You can see the emotions of the characters, and overall the composition underscores the significance of the philosopher's death, especially in the context of the enlightenment/romantic period. (This is my opinion as a pleb; not an art expert.) But what I do understand, as a computer person, is that these concepts are still beyond what a model can encode in an image as of today.
So although humans are no longer superior in the mechanics of producing images, I think in the higher-level/psychological aspects of "art", there's room for humans, at least for now.
And although I'm just an dumb anon on the internet spewing these ideas, I know I sound harsh, but I think I'm correct. Because the reality is that now, everybody's downloaded the SDv1.4 weights onto their hard drives, and the cat's out of the bag permanently.
I have never used any of those services before. I have never considered using one of the services before because it was always outside my price range.
Now that I have tried stable diffusion and have photo bashed and rendered some concept art for each of the main characters in my novel, I now want to commission an artist to create the 30 or so needed training images so I can ask stable diffusion to spit out my own character in various poses and expressions.
At minimum, that will require a human to render a model sheet of the character from front and back and side and 3/4 and above and below, as well as the emotions on the basic emotions wheel.
If I want the character to be able to wear different outfits, then I will also need to pay for renderings of that character wearing that clothing, all in service of trying to train stable diffusion to be able to remix that character into future images.
Let's also say that I do not have a killer graphics card to be able to train images into a model, luckily, for another $100 fee, the artist will use their existing graphics card to spit out an embedding or a hypernetwork or a VAE or whatever it is that you can use to add custom training to a model and send me that as well as the original set of input photos.
After all of that, I can generate the photos I want...but I will then have to slightly tweak each image so that it has human authorship, even if it's just removing noise and fixing the cursed loops that happen on limbs at times.
In short, I am considering something that is at least $200 for the crappiest cheapest artist out there, multiply that by my six or so main characters, and that is money that I am genuinely considering spending that I would not have even dreamed of entertaining for a moment.
The proof is in the pudding as to whether this thought process will be happening for other stable diffusion users who are able to get images they like, but do not have good rendering skills on their own, weather they too are willing to pay for this or not.
If so, there will instantly become a new type of artist job available, that of the AI art trainer artist.
At the very least, there will be AI art cleanup artists that remove the noise and so-called cursed elements of ai art when used in the concept art stage.
Currently, we (mostly) still need people to make the tapes. That probably won’t be true forever, but it is for now.
People on HN have very strong options about this stuff without looking at any of the relevant case law.
And why should that be a problem? This is a King Canute and the tide scenario. We may as well bow to reality and admit that it is ok.
Now it’s so easy to share music, it’s meaningless to try to stop people doing it.
Apply 20 years of law cases and punishment for random people and now…
…everyone pays to stream their music.
Right? Wrong? Eh.
I’m just saying, you are kidding yourself if you think that the Powers That Be will just let people decide copyright isnt a thing anymore because of (insert reason here).
Once there is money involved, there will be court cases, and you know, I’ll be shocked if a combination of “needs bigger GPUs to run” and “legal issues” don’t cause these sorts of models to be locked away behind cloud APIs in the future.
It is what it is. Enjoy it while you can; some things are quite predictable, and:
“Law takes a while to catch up with new technology, but it eventually does, and when it does it favours the status quo”
Is one of those things.
Not always; eg. Uber, but… predictably often.
I will point out that the usual copyright maximalists have been pretty silent on the issue of AI art. The biggest opposition to AI is coming from the Free Software community - i.e. the people who want to abolish artists' ownership over their work outright.
I'm surprised here; I thought it only cost like $600k to train SDv1.4. Plenty of wall street folks and programmers have that much money to burn.
You don't need to pay to listen to music if you don't want to. The corpus of good music on YouTube for free is probably bigger than what you can listen to in a lifetime. If it isn't already it will be in time.
Napster's model won that war. Effortlessly. If you're paying for your music, you are paying on your terms based on the value you think is being provided to you. It isn't a legal framework making you do it.
My point is that Napster-to-SD is not a fair comparison. Napster didn't eliminate or replace human artists, while SD certainly did. Therefore, it's not fair to assume the government can regulate themselves out of this one like they did with Napster. Because even though Napster enabled widespread piracy, the human musicians still had leverage in that they were needed to create new music.
So my answer to the question posed by parent comment of "will SD play out the same way that Napster did?" is "no" because there are fundamentally different economics at play.
Obviously, if a good generative model comes out, the music industry will be in a similar boat to the art industry right now. Google was working on it in 2017 (https://magenta.tensorflow.org/performance-rnn) but I don't know if they've made any progress since.
I'm going to need to see a source on ripping the same color palette as being illegal.
I bet Pantone believes they can copyright a color palette, considering Adobe just pulled their colors out of Photoshop.
The other is amusing in light of this case: https://www.supremecourt.gov/DocketPDF/21/21-869/213328/2022...
Unfortunately art is a hard business to make a lot of money in and the vast majority of the people unhappy about this do not have the money to mount any kind of legal challenge to the corporations developing these databases, so we are basically fucked.
Which is in some ways nothing new, the average Internet user has no concept of creator's rights and will blissfully assume that if something's not behind a paywall, it's free for any use. It feels even shittier and worse when this is being done on an industrial scale by these AIs, though.
Open source models will likely be dominant (this was how Stable Diffusion got popular, it's not clear their new neutered 'safe' model will be as popular or a successful business).
This is the new reality even if the bigco gets sued out of business. Artists need to accept that reality eventually. Just like all "panics".
Fortunately AI generators are not a direct replacement for artists, I've seen it used as part of the design process, but that still takes talent. Maybe it will get better but it's not a one-stop shop for business use-cases. More likely it will be AI+artist, not AI replacing artist.
A real open-sourced model trained on images that were explicitly licensed for such uses would be interesting. It would cost money but it would also not be hiding a ton of its actual cost by training it on images whose creator never imagined that "being dumped into a vast dataset" would be a thing that would happen, as well as images that were fine in their original context as "fair use" but got sucked in by the web crawlers feeding the dataset. You want to train an image-generating machine on my work? Ask my fucking permission and if I say "pay me" then you either negotiate a price we can both live with, or you do not put it in your dataset.
I do not accept your reality. Sue 'em all until you've gotta train your own damn datasets, or buy ones that have worked out the proper licensing. If Photoshop can be made to refuse to scan currency, then these things can be made to be fair to the artists whose shoulders they rest upon.
One might argue that any unlicensed use of copyrighted material is misuse (unless the courts have ruled it is fair use.)
Only the copyright holder may bring action against a potential misuse. Typically, individuals tend not to have deep enough pockets to hire the lawyers to bring the action. And during discovery it may be found that the model was trained on works-for-hire made by humans replicating the original style. Or appeals may continue on at more legal cost, without guarantee of recompense.
Most will decide it’s just not worth it. Until a company similar Getty buys up all the copyrighted material and, in addition to sensible suits, brings spurious legal action against individuals and turns the tables.
Court cases / legal clarification may outlaw Stable Diffusion or DALL-E for having been trained without the consent of the original artist on their copyrighted artworks. But if this technology proves valuable and viable, the likes of Disney, WPP, and Omnicom Group will pay a few dozen artists to create enough work to seed an engine that can generate 50 million wholly-owned lookalikes.
Copyright protection will ultimately shield the megacorps from competition by the common folk, not artists from competition by corporations.
No, it's only legal because nobody has litigated a case all the way to the Supreme Court. We all thought that APIs weren't copyrightable, and then, suddenly they were (or worse, were "assumed to be" but not explicitly ruled upon).
In neither case is it legal to use outputs of an AI that infringe an already-existing copyrighted work. This means anyone using it to generate production-ready artwork is exposing themselves to potential legal liability if their AI winds up regurgitating training set data. This is what I worry about way more than just "are the model weights infringing copyright".
As for morality:
- Absolutely none of the current generative art systems are trained from scratch on ethically-sourced datasets. The AI companies just assume that because they need ungodly amounts of training set data, that they are morally entitled to get it, because they spend more time staring into the eyes of a basilisk[1] than worrying about the world they already live in.
- Multiple companies getting into generative art have gone above and beyond in giving human artists the middle finger. DeviantArt decided to blow all their good-will from defending against NFT nonsense by making their AI training program opt-out, which is NOT HOW CONSENT WORKS. Mimic[2] and Dreambooth are basically artistic impersonation tools that have specific moral implications beyond the general practice of AI art.
Whether or not this becomes actual law is... well, I'll put it to you this way. Stability AI's Dance Diffusion is actually trained on an ethically-sourced, public-domain dataset. Why? Because they're afraid of being sued by the RIAA. Despite the name, copyright maximalists are less about maximizing the rights of artists and more about building moats around large publishers. And generative art does not threaten[3] those publishers, so it will not be banned.
[0] Yes, the same one that mandated upload filters on video sites.
[1] Roko's Basilisk posits the idea of a superintelligent AI - one that can eat everyone's brains and emulate them perfectly - constructing a perfect hell for people who didn't build it as a way to threaten those people in the past to build it.
Longtermists can burn in computer-generated hell. Time discounts exist for a reason.
[2] https://illustmimic.com/en/
[3] Specifically: even in a world where AI completely outperforms humans on all artistic endeavors, publishers will just buy the AI companies.
Same with copilot; people copy shit from GitHub and SO all the time without mentioning copyrights and a lot of code people cough up is just ‘stolen’ from someone without remembering who/where it was; what’s the difference?
Lots of bands (nearly all?) learn their craft by playing covers. At some point the artists in the group start to find their voice.
I suspect AI never will.
They are both end 50s, so I rather doubt it.
What does this mean?
Barriers to entry? Training and hard work are barriers to entry? So the next time when you go to the dentist, tell him that you can fix your tooth with a text prompt.
Yes, assuming there is no substitute.
I agree with the problem, on platforms like AWS you'll even need to send a manual request so they would let you use instances which can run SD. On the other hand there's already something like replicate.com, which allows you to run the SD like an API. I hope there will be more services like this.
https://replicate.com/blog/run-stable-diffusion-with-an-api
In my pitch I proposed generous royalties and cash compensation to the artists. All of them essentially said that they weren't taking commissions. I'm sure this is in part due to the fact that I have never been involved in the graphic novel world, so they'd be taking a chance on someone new. Still, it seemed there was no amount I could pay to get someone to work with me.
Now I am reviving this graphic novel idea and still looking for someone I can pay to help me work on the project. However, the more synthography advances as a technology the more it seems like I should explore it as a path for this project. It would help me develop the book much faster than I can on my own. At minimum it could help me get the story board in place.
Maybe some day synthography can help artists scale up their output, in the same way Michelangelo developed a workshop of apprentices. I would be more than happy to pay an artist's AI apprentice if I can't work directly with the human.
How is art defined? How is it defined by artists and how is it defined by consumers. Some artists may want to define it such that it's made by people. Much fewer consumers would make that a requirement (aside from some art collectors). Consumers will care much less of how some art was generated and they will dictate demand.
So they're taking about "animation"?
This and image-to-image generation (more in a moment)
A reader will see there's more writing, you don't need the parenthetical.
I found this funny as well, but on reflection I think the author is talking about the specific effect of changing the seed but not the prompt. To me it highlights a weakness of AI imagery — no intent, no progression, no story.
Eh, yet. I can see where the prompt just gets extended to, "in a hilarious series events in the style of buster keaton" and all of a sudden your vampire-toothed anime furry in cyberpunk clothes is flying down the side of a building, being saved at the very end by a sunshade they flop into. But I don't know if I'm going to live long enough where AI makes new Buster Keaton shorts using commonplace elements around them in new and exciting ways. Like would AI know that you can hang from the hands of a clock, which move over the course of a day? to me that's some creativity that was years ahead of its time.
Now that I've attempted to predict the far-from-now future, I'm sure we'll see this in 6 months.
Digital artists doing commissioned work in the style of some other more famous artist? Bespoke stock photographers?
It seems these jobs will simply use the new tools, and sell a more manual version when needed.
Similarly, AI-remixed¹ music is getting better and better. I want to know how the author will feel when their music² is used to build a model that allows anyone to create infinite music in their style and beyond, much of which will be enjoyed far more widely (because it will be free or royalty-free) than the source material ever will.
¹ It doesn't seem fair to call it "generated" or "created" when it'd be useless without an enormous corpus of source material powering the thing.
² https://vegadeftwing.bandcamp.com/
Right now, your music requires attribution and can't be used for commercial purposes.
Should visual artists have the same ability to choose the rights they reserve?
Sure, but fair use still exists. If someone takes just a single drum hit, reverses it, and coats it in distortion and reverb, but some how I find out they did it, I can sue them. But I can sue over anything, that doesn't mean I'd win. I'm pretty confident they would, and would agree they should because it's fair use. I think training an AI model is fair use as well.
> Should visual artists have the same ability to choose the rights they reserve?
Yes, and within the bounds of fair use as musicians that I've stated above. If I photoshop out a single eye, let's call it 50x100 pixels of a 5000x5000 image, and use it to make a neat mash up art, I think that is transformative enough to be fair use. Similarly, I think the AI model is transformative enough.
In either case, there are limits. If someone makes a model fine tuned on my specific music, that's wrong. Similarly, if someone makes a model fine tuned on an artists specific work without their permission, that's wrong. Using an artist name (music or drawing) in a prompt is gray, but I personally think leans on being acceptable, especially if that artist is dead.
As for me, I just feel that it should be looked at the same way that photography is looked at; that it's a new medium all to itself. You wouldn't enter a photograph in an oil painting competition, so you shouldn't enter AI art into a drawing/sketch art competition either. It would also be useful if there was an acknowledgement that AI was used in some elements of the art, but like as not there will always need to be a human agent that makes corrections in what is created.
I understand that photography generates waste images. I just don’t feel like it’s the same.
In my view, in the past it was believed that to be an artist and produce the output you admire, you had to learn things. You had to be patient, and spend thousands of hours learning a skill. But I don't think it starts and ends with just 10,000 hours of brushstrokes. All that time and effort likely causes other neurological changes owing to the situations that being a traditional artist forced you into, like being exposed to other people and communicating with an audience.
The ability to generate art with prompts bypasses all that time and effort. Using Stable Diffusion, I have made incredible pictures (that I don't consider art), but at the end of the day I still feel like just another person chasing yet another rush of dopamine, the same as it's always been. It's an enabling device. I have never been able to summon the will to practice drawing or painting for enough time to see results, and that hasn't changed. The only difference is that I have the end results, which is mostly what I wanted in the first place, and it's pretty much all I'll ever get with this setup.
But it is often repeated that "focusing on the end results" when trying to learn a creative skill results in bad outcomes. It is a character flaw that someone has to overcome to get better. Now I feel that 70% of the road to "the end result" can be bypassed without the need to work towards that additional knowledge and virtue.
When artists speak of AI damaging the integrity of their profession, I think this is part of what they mean. For the first time in human history, it is now possible to be a better "artist" without growing into a better person, or developing a wider-scale notion of what existence or the world outside is. The folk assumption that growing up as an artist changes you in fundamental ways (by forcing you to study the craft and observe the world for what it is) has been violated.
The people who treat art as a profession or hobby to work at over thousands of hours, for example, will probably share a lot of significant interactions and bondmaking as their skill gradually improves over the years; not so with a person that can generate pseudo-masterpieces within minutes.
I'm going to guess that the long-term downsides of AI art are going to be baked into cultural norms instead of the limits of silicon. There has already been significant backlash from art circles when tech-oriented people outside of the scene train models in the style of people like Kim Jung Gi, who had just recently passed away. Many people's rules about "art" have gone unwritten for so long because this current situation was unthinkable only a decade ago. Regardless of intent, being seen as plagiarizing the style of someone who is no longer alive isn't a good look.
Still, what I always want to keep buried in my subconscious about how AI art is used and treated is that my insatiable interest in the subject is fueled by my lack of (ability/drive/bravery) to express myself through my own actions, whereas my embarrassment is spared by delegating to a machine to do the work for me and guess what the inner workings of my imagination look like. If this is something I need to overcome, it's only going to get harder for me to do so as the state of the field advances further.
The word that keeps coming to mind when I look over the things I generated is "intent". It doesn't feel like I "made" the things I did, but that I "found" them. There's not a lot of insight to be gleaned except the keywords that each subset of popular culture finds the most useful or interesting in regards to prompt engineering during a particular timeframe. My preo...
This is a big part in why I enjoy vastly different kinds of music creation. When I work with a traditional DAW and build a track from scratch, I'm sculpting and crafting an artwork. When I work with modular synthesizers and generative compositional tools (AI or not) I feel like I'm navigating sonic landscapes to find interesting destinations.
AI art is the latter, and while it's fun, it's a different feeling during creation all together. In music, I can typically hear which the artist was doing in a given song. With Stable Diffusion and DallE, I don't know that I can as reliably. This is why the divide is so interesting to me. We have to radically different avenues, each with their own merits in how fulfilling they can be, to choose from which produce similar results.
I will admit, there is something to the reward of making something hand crafted that at least to the artist, adds value, even if that value isn't (typically) transferable to the end user.
Like, this isn’t some kind of hack’s approach to art. Literally every working lyricist that I know uses a rhyming dictionary from time to time. Why? It’s fucking work! There are deadlines!
With regards to AI visual art… the best results I’ve seen have been from people with strong visual art skills. Aesthetic choices still matter a lot when you’re tweaking the prompt, model, hyperparameters, etc.