Maybe this current explosion in the relevance and visibility of this kind of AI model will finally lead us to rethink how insanely nonsensical our IP systems are. I'm not holding my breath, but there's hope that this sort of thing will (combined with situations like the HBO debacle) clarify the need for massive IP reform in the cultural zeitgeist.
The problem here isn't that the model was trained on copyrighted works, the problem is copyright itself and a cultural focus on collecting rent. We probably need UBI or equivalent to deal with the outcomes here in a healthy way.
Why do people keep bringing up copyright anyway? It seems pretty clear that the images being generated by StableDiffusion are transformative so it's protected under Fair Use.
Yes but we have a common law system and there's already tons of precedent that training AI systems is transformative.
It's also quite obvious just by looking at the generated images that it's clearly transformative. The images generated are unique and you can't trace the original copyrighted image from what's generated.
You really don't need a judge to see that Fair Use covers Stable Diffusion.
Federal lawsuits are not cheap and the default is that you pay your own costs, you have to win the argument that you should get court costs & attorney's fees.
What happens if you give an image prompt like "mona lisa", "daffodils van gogh", or similar designed to describe an image the model was trained on. Will it generate that image?
Or for written works, start with a sentance from a copyrighted work, or part of licensed code. Will it start reproducing that work word for word (like code pilot can do with the GPL license)? Getting these to generate copies of GPL'd, company owned, or other code with restrictions can lead to complex issues for the person/company using that code. Or likewise if a story contains significant elements of copyrighted works; worse if the works have trademarked elements.
> The images generated are unique and you can't trace the original copyrighted image from what's generated.
That might not always be true. I've gotten some results back that had the Getty watermark on them and others with the artist's signature. Unless the AI is adding that to images that never had one before (which might be a trademark issue), then you might be able to determine the provenance of the image components.
Is there something equivalent to the yellow dots printers add to their output that would survive the AI transformation?
Well, you can still be sued even if the reason is almost totally bogus, and you have to go to court and make a stand that the reason is bogus. Doing only 100% legal things won't protect you from being sued.
There are really significant, novel copyright issues implicated by these large generative models trained on other people’s IP.
If you take a step back, you can see that there are different ways to frame what is happening. One frame is: “Defendant built an algorithm that memorized features of Plaintiff’s IP. Defendant’s algorithm recombines parts of those features in order to produce works in the same domain that compete with Plaintiff’s work, all without Plaintiff’s consent.”
Bear in mind that copyright holders are among the most litigious out there. If generative art becomes as big a deal as some people expect, they will have every incentive to use their huge litigation budgets to claim a piece of the action.
Because the law develops very slowly, the legal process has not yet had the occasion to really evaluate what transformative use means in this novel context. I’m personally interested in seeing where things go, but it’s going to be a while before we know where the law is headed.
> “Defendant built an algorithm that memorized features of Plaintiff’s IP. Defendant’s algorithm recombines parts of those features in order to produce works in the same domain that compete with Plaintiff’s work, all without Plaintiff’s consent.”
The fun part is, this is how human artists learn too.
They absolutely do. What they don’t do is mechanistically clone compressed mathematical representations of input data. The human part of the creative process could very well be a distinguishing feature, legally.
It will be hard to tell whether an image was created by a “human creative process” or AI.
Will the same image be legal if a human made it but not if it was created by Stable Diffusion? How will someone even know, short of a legal discovery process?
A legal discovery process is a perfectly reasonable (and common) way to determine if something is legal, and yes, it's completely unsurprising that the exact same outcome might be legal or illegal depending on how it was obtained (or even with what intent the actions were done), legality takes these things into account.
Tracing and collage both mechanistically clone elements of the input. Trendfollowing, mimicry, and copying others is standard in any creative area.
I don't think computer-generated works will be easily distinguishable from human unless they're desired to be or shipped with metadata. It's already hard enough to distinguish human artists from other human artists without having names attached up front.
> Tracing and collage both mechanistically clone elements of the input.
Yes, and tracing counts as art fraud.
Collage is a bit different, because you are mixing many clones of many other objects such that you create a new object; additionally the way you assemble the clones may transform them (a photo of the mona lisa has different surface texture than a painted version, even more different if it is clipped from newsprint), but while the borders of this are not clear, it is clear when people are far enough over the border. Think of hip-hop, sampling, and remixing music, and some of the legal battles which have come out of that.
They can (and do) clone compressed electrical representations of everything they see. Everything is just stored in memories in their brains instead of memory on the cloud. In the case of these complex AI, you cannot really extract the initial works, right? They have all been atomised, mashed together, and very imperfectly encoded as weights and parameters.
Yeah I am sure a lot of lawyers are going to have a lot of fun arguing every way imaginable.
I see this type of reduction again and again to advocate for one position or another relating to ML. Human consciousness and thought processes aren’t “just” anything (math, electrical impulses, etc.) — the fact is, we don’t know what the brain really does and how it’s connected to our conscious experience, or even what that is!
Deep learning is very powerful and impressive in its applications to date. However, it’s so saturated with hype (and humans are so prone to anthropomorphizing things) that it’s often viewed as something much more profound than it actually is. Neural networks, despite their name, don’t model the brain. And they lack a whole array of “intelligence” features that humans possess and use constantly.
All of this is to say that there are very significant differences between computer algorithms and human cognition, and I tend to think the legal system will be unpersuaded by arguments that ignore those differences.
Also, this is to say nothing of the public policy interests that shape the law. Regardless of what’s “under the hood,” the law can simply treat human and machine output differently. I’m not a copyright lawyer, of course, so I can’t speak to the norms or technicalities of copyright law itself.
> Human consciousness and thought processes aren’t “just” anything (math, electrical impulses, etc.) — the fact is, we don’t know what the brain really does and how it’s connected to our conscious experience, or even what that is!
There are a lot of things we don’t know, but it is not magic. There is no discussion that artificial neurones don’t have much in common with the real ones, which are very non-linear and much more connected. But in the end it’s all electrochemistry.
> Neural networks, despite their name, don’t model the brain.
But that’s not directly related to my point. My point that even in the case of a ML model, you cannot get an exact reproduction any more than you can get from a human’s memory. In one case it’s scrambled somewhere in someone’s brain, in the other on a hard drive but the difference is not really relevant. Subjecting an AI’s production to the copyrights of all the things it’s been exposed to is very similar to subjecting a painter’s production to the copyrights of all the painting they have seen.
> Is my brain's floating point calculations subject to round off error?
>
> In the (actual) neurons, is there a representation of real numbers? Where are the numbers in the brain stored?
None of these details are relevant to the bigger picture similarities of non-hardcoded learning from training data. None of these details change the ethics of what's being discussed here.
> I feel like people who assert this neither understand what neural networks are and how brains work.
> The fun part is, this is how human artists learn too.
We don't actually know exactly how human artists learn, and human artists are capable of innovation, nobody knew pointillism or Bauhaus before they were invented.
A little know fact is that for humans it takes a long long time to learn, while they learn, they develop a style, if they don't they are not "real" artists, but merely executors, artists evolve, sometimes dramatically, in unexpected ways [1] [2].
So for us humans learning is an experience, not just recombining parts of features of other things.
We are also highly influenced by feelings, unfortunately, so sometimes we do things a certain way because we felt that way, not because we wanted to paint that thing that way, or because we are not good enough to do exactly what we wanted to do.
Is Mona Lisa happy? Who can tell?
Was Leonardo happy when he painted it?
What was Leonardo thinking when he painted it?
What was happening in his life?
Is that the best smile Leonardo could paint or it's an enigma he put there for future generations?
These questions are more important for an artist than the mere features of the painting.
The philosophical question is: is art discovered or invented?
If it's discovered, then SD can generate art, if it's invented, than SD it's not even generative work, because to invent something from something else, you need inventiveness.
Artist today use the exact same method of learning from other peoples artwork to generate new artwork and styles. These models are learning just like any artist learns and then producing new content.
- It appears people can train AIs from scratch or at least fine-tune them at home.
- Even if your art isn’t in “the training set”, that does not prevent the AI from learning its style. (Someone can decode it to CLIP embeddings. It could have a really good text model trained on vivid art museum descriptions of your art.)
- The ability of an image model to generate your art means it could also be trained in reverse to recognize it, producing a caption model, which would give vision to the blind. And surely you’d feel bad about that.
If you forbid it, the investment in developing those models disappears. They will be stuck at ~what we have now at best.
You can also require cloud providers to enforce a ban on training (and deploying) such models, it's doable. Good luck training it in your basement, it will probably take you a decade.
If this is banned, it will become a lot like piracy - yes, it's available, no, most people (at least in the West) don't do it, practically no businesses do it.
Training these models is much much cheaper than you think it is, and there’s good data for it already.
Either use a CC0 set like Wikimedia/Flickr and throw in some dead artists like Brueghel, or train on data from a country we don’t respect the IP of. Lots of Taobao product photos out there. It’s enough.
You are getting ridiculous now. Training this on "Taobao product photos" will lead to a useless model that is unable to produce practically all of the "cool" demos posted here in the last week.
A few months ago this task was virtually impossible. Then it was possible, but extremely expensive and pay-walled behind the "Open"AI' website.
As of about a week ago this tech runs on consumer GPUs. The weights have been downloaded 100s of thousands of times, and fine-tuning / modifying is possible.
Training from scratch is about $500k still, but it will only get cheaper and easier.
This doesn't contradict anything I have written. The average technical user will be unable to train this exact model (not to mention the supposed future more powerful ones) in their basement in this decade.
That just feels like such a pessimistic forecast to me. Of course, the current trajectory of improvements in model efficiency and better commercial GPUs / ML-accelerators may hit a wall.
But I would not be surprised if this was trainable on a commercial GPU at home within that time. But I think another important trend that we are seeing is that you don't need to train these models from scratch.
Open-source "foundation models" means that you can usually get away with the much easier task of fine-tuning, as to not throw away / re-learn everything that these large models have already fit.
Edit: I initially said 2-5 years, but on more reflection this does seem optimistic (for training from scratch).
If things go that way, the 'legit' models will continue dev, just using licensed content (along with public domain works). It will be more expensive for the end user, but that cost will shrink over time for general work. Tools that mimic working artist though might not be available (or will expensive). This all seems pretty ideal, so the pessimist in me guesses it's fairly unlikely.
I am not sure it will be possible to get enough training data that way.
I don't know enough about diffusion models but if LLMs (of current size) have to use only public domain, they will be undertrained and we will see significant degradation in performance. Not to mention that Codex will be effectively dead.
> It's absolutely logically consistent to allow humans to do it while forbidding AI to do it.
Why would you, though? If art is for the sake of art, then all art is valuable regardless of origin. If art is for the sake of providing human employment, AI being better in no way stops performative make-work from existing. If art is for the sake of copyright trolls to troll harder, then fuck art, feed it to the AI!
Art is for the sake of humans producing said art. I don't give a flying f*ck about "art" (it's not art) generated by a neural net using statistical patterns.
This is such a strange position to hold. At some point in the future may be confronted with an image that is emotionally stimulating, and you will have no idea whether a human or AI created it. Are you suddenly going to dismiss and discard it just because you subsequently learn it was AI generated? You must see how silly that is.
Your "logic" for making AI art illegal is basically "don't like it". Your personal and subjective opinion is that it's not art by definition.. This is like refusing to eat artificially grown meat because you have some strange idea about what food "should" be. Even if the meat was made MORE delicious you would still claim it wasn't food and turn it away. There's no logical consistency to your position, it's purely reactionary.
My position is perfectly logically consistent. Art is distilled human experience and human emotions emerging in a particular context after a chain of events. None of this is true of AI "art".
What is not logically consistent is to claim that a black box utilizing statistical relationships between pixels in a giant dataset is an "artist" and that its products create "value".
The compiler is not a programmer, AI can never be an artist.
A good chunk of art is just that, to my understanding. People go to art shows with a knowledge by whom, when and “how” the art was done. They will be very confused if asked to tell two pics apart if you time-travel to Picasso, ask him to paint a new unique pic and then generate another one with AI. They can even find an idea, symbolism and what he felt/thought of behind an AI version.
All this boils down to a simple fact that egos like to think of themselves (and of artistic interaction) much more than there actually is.
I remember a story when a literature teacher insisted on a definite symbolism of some minor detail in a novel. People contacted the author about it and he said no, there is nothing behind it. It was just a filler without any second thought. Makes you think how much symbolism is far-fetched in classics, where you cannot simply email an author.
Human artists can copy an art using transparent tracing paper on a lightbox just like that, and that will be plagiarism. No reasons AI output shouldn’t be.
This is patently, obviously _wrong_ for anyone who has tried learning any artistic skill in their life. Sorry to be this straightforward, but it gets on my nerves every time I read it.
If you tried learning, let's say, the chiaroscuro technique from Caravaggio you'd be analyzing the way the painter simulated volumetric space by using white and dark tones in place of natural lighting and shadows. You wouldn't even think of splitting the whole painting into puzzle size pieces while checking how many how those look similar when put close one another.
Given somewhat decent painting skills, you'd be able to steadily apply this technique for the rest of your life just by looking at a very small sample of Caravaggio's corpus.
On the other hand if you tried removing even just a single work from the original Stable Diffusion data set you used to generate your painting, it would be absolutely impossible to recreate a similar enough picture even by starting from the same prompt and seed values.
Given how smart some of the people working on this are, I'm starting to believe they're intentionally playing dumb to make sure nobody is going ask them to prove this during a copyright infringement case.
>This is patently, obviously _wrong_ for anyone who has tried learning any artistic skill in their life. Sorry to be this straightforward, but it gets on my nerves every time I read it.
Both my parents (though retired now) were commercial artists. I was trying to be an artist at one point in my life before moving in Engineering and Science. All my parents friends are artists so I grew up around artists.
Ask any artist here who is using illustrator, Photoshop, Krita etc. How often do they google image search for textures, or reference images that gets incorporated into their artwork? The final artwork is their own but it may incorporate many elements from others artwork.
>If you tried learning, let's say, the chiaroscuro technique from Caravaggio.. You wouldn't even think of splitting the whole painting into puzzle size pieces while checking how many how those look similar when put close one another.
Ever seen hyperealistic pointillism?
Who are you to be the arbitrator of how an artists creates their work? Have you ever gone to a modern art gallery and seen all the different methods people use to create artwork?
Art is boundless and unique to each who creates it.
If a Artist uses a tool to create art, everyone agrees that is art. It could be a paint brush, clay, software on a computer etc etc. If an artist uses AI as a tool to create art then suddenly it's not art.
Whether a work is transformative or not isn't the only factor that goes towards a fair use ruling. Bring transformative is just a part of the "purpose and character of use", which is a single factor weighed alongside the nature of the original work, the amount and substance used (if the "heart of the work" was copied), and the effect the derivative has on the market for the original work. It's much more complicated here, and will have to be ruled on a case-by-case basis, as far as I know, and copyright holders could potentially make many cases using the other factors that a use isn't fair. I do think it should be considered fair use, but as the law currently stands, it does appear to be more complex, at least from my point of view.
The new technology looks like it could threaten artists' livelihoods by replicating something about their work. The old legal framework for protecting artists' livelihoods from photocopiers etc was copyright. The goal of making sure artists can both make art and eat is still relevant, so people reach for the tool that used to make sure that was possible.
If it really does make it hard for artists to eat from selling art, that to me is just sour grapes. I mean there use to be 1000s or 10s of 1000s of liveries to lock up your horse. There's no goal to keep horse keepers so they can still keep horses and eat. Instead, their industry mostly disappeared. I'm not heartless but at the point their services are no longer needed that's just the way it is. Cloth weavers replaced by the loom.
I don't think SD will replace most top artists for now. It's hard for me to believe SD is going to come up with images like those from top concept artists. But I can imagine SD replacing lots of situations, like maybe stock photography, when you can just ask the AI to draw "people in front of whiteboard discussing sales chart"
We need a different tool than artificial scarcity. Both for copyrights and patents. Government funded prize systems and patronage systems with a lot of mechanisms for citizens to choose what to fund seems ideal.
Online artists esp. fanartists have a strict moral system with rules like “credit the original artist” that isn’t based on actual laws, so they’re upset about this.
I think that's wishful thinking. It's objectively a mechanical derivation based on many copyrighted works. In many cases it's going to reproduce some works with not that much transformation.
What's a similar thing that has come before this? I can't think of any, this is very novel. You'd want to wait for some rulings before you jump to conclusions.
It might be similar to some forms of (human composed) sample-heavy music which use bits and pieces from many different songs to create something entirely original.
As far as I understand it, this is still considered copyright infringement in most IP law systems. (If the samples aren't cleared)
Fair Use is a defense, not a right. Just being transformative isn't enough here, that's just one of many different factors that needs to be checked. Furthermore Fair Use includes evaluating the effect an infringement has upon original work's value.
So when your image generator keeps spitting out Gettyimages watermarks, while you are building a service that is in direct competition with Gettyimages for stock images, there is an argument to be made that Fair Use really doesn't apply here. As what you are doing is essentially stealing Gettyimages' work, AI laundering it and selling it back to their previous customers.
With StableDiffusion a Fair Use defense might have an easier time, as the results are released to the public. But it's still not exactly clear cut. If you type in "Mona Lisa", you'll still get something that looks like a copy of the Mona Lisa, not like an original work.
Creating and allowing public use is a positive externality. Negative externalities should be taxed, and positive externalities should be subsidized.
Everyone should be able to freely use any public works, and governments should ensure that people whose art is being used to train these models are able to continue to do so.
How about some company just make site that licenses art. In secret they run SD to generate images forever. They use image search to search for similar images on the net and claim copyright. PROFIT!!!
What will happen is AI will be fed IP legal corpus as part of training. It will only show images that can't be linked (in a justice setting) to any particular work. Which will be interesting because it will produce media culturally alien but still appealing and probably addictive. It will literally be the engine of advancement of human culture. I'm not the conservative type, but I'm still slightly concerned as much as fascinated.
>What will happen is AI will be fed IP legal corpus
I first read this as a prediction that AIs will be employed to generate future IP legislation, and now I don't know if I'll be able to go back to sleep.
After using SD heavily for a week, I half agree with this. It is incredibly disruptive, and it's wild how much it accelerates the creative process. I'll give you that.
But two things I've noticed:
First, artists will still have a massive advantage over non-artists with this tool. A photographer who intimately knows the different lenses and cameras and industry terms will get to a representation of their idea much faster than someone without that experience. Without that depth of knowledge, someone might have to rely instead on random luck to create what's in their head. Art curators might be well-positioned here since having a wide breadth of knowledge and point of reference is their advantage.
Second, we need the ability to persist a design. If I create a character using SD, I need to be able to persist that character across different scenarios, poses, emotions, lighting, etc. Based on what I know about the methods SD/Midjourney/Dall-E are using, I'm not sure how easy this will be to implement, or if it's even possible at all. There will always be subtle differences and that's where being an artist who can use SD for inspiration instead of merely creation will retain their advantage over a non-artist.
I haven't had a chance to try it yet, I'm optimistic but skeptical that it can have true persistence like I'm referring to. Especially since it requires training your own model which requires multiple images of the same asset/character/object/etc.
For what I think you have in mind, I suspect it will eventually not be "image to image", but "<ai thing> to <ai thing> + image", for that guess to be remotely repeatable. That "ai thing" is probably the persistence you're talking about.
I think that thing will neccesarily contained representations of dimension, behavior (physics/bones), and "style". Without the "ai thing", if only using an image/text, the character would have to be impossibly represented in the model, so it could guess all of these things predictably. For example, what that character looks like from a side profile, or behind. What if it's an alien, and its arms should always bend backwards? Could a text representation ever be made to completely describe this, with good reproducibility? Probably not. But, I assume some non-human representation would have a better chance.
As is, if something known is required, I think the behavior of these models can be considered "destructive" to the input image, more often than not. For this reason, I think artists are safe, for the time being. :)
For face generation, I think there are deep neural networks that can generate multiple views of the same face [1], [2]. Stable diffusion already provides the possibility to generate variations. So I don't think it is a stretch to imagine that these existing capabilities will only get better and/or be applied to SD.
[1]: Multi-View 3D Face Reconstruction with Deep Recurrent Neural Networks
[2]: Deep Neural Network Augmentation: Generating Faces for Affect Analysis
What about using this tech for ideation and artists for production?
You could use Stable Diffusion et al to create new characters based on a prompt, then farm the concept out to artists to produce individual works. Kind of like hiring a super expensive agency to design your new logo or brand identity, then using a stable of in-house designers to translate the concept into UI, ads, etc.
Honestly, I don’t even know if we’ll need humanity for the final product. It’s like using a chess computer to get an idea of a good move… and then a human to approve it? Adjust it? Be inspired by it?
Any signals humans give (painting x is better than y) is another signal to encode. Take billions of such ratings and improve the AI’s taste to superhuman levels.
In short, anything that humans would add to artistically improve the outcome is just another signal to be encoded. It’s weird to write it, but artistic creativity is deciding what new pixels go where, which is a search problem (in a large search space) which AIs are apparently doing great at.
We have a bias: we’re humans, we must be important somehow! But it comes down to a bigger neural net eventually outperforming the one in our heads.
The difference is that chess rules are extrinsic to the players at the board, whereas art is intended to communicate (emotions, narrative, ideas) to humans.
A neural net that can communicate with intent to human minds as well as a biological human is nothing short of strong AI. We’ll get there, but not with generative models.
The datasets these tools use don’t include any context. There’s no sense of what the images in the data might mean to the viewer, or how they relate to the time and place they were made. I would argue that means the tools will struggle to produce meaningful works, even if they become great at making beautiful works.
> It’s like using a chess computer to get an idea of a good move
They did this for Backgammon and found styles of play humanity hasn’t discovered in millennia. Now humans can use those when playing each other and it makes an old boring game feel fresh and exciting.
Useless? One can spend days/weeks/months enumerating different concepts of a design due to a roundtrip between “maybe we try <a prompt>” and an image. Now it’s literally minutes and a designer can draw you some ideas right at your office.
For every tool, a better output will always be created by people who specialize in a craft. Just as photoshop revolutionized photography, to this day you can tell the difference easily between and bad 'shops.
In video games upon release everyone is bad and its an even playing field. But as people practice and gain experience they improve their usage, refine their approaches. You eventually see metas develop and best practices.
This will all be exactly the same. Just give it some time and watch professionals create professional outputs compared to paying for some base level work on Fiverr or whatever.
There's one huge difference between Copilot and stuff like this. Art that's 98% correct is awesome. Code that's 98% correct is completely useless.
I think Copilot is going to live off hype for a while then tank and be looked back on as a failed experiment. Whereas I think that this kind of AI will eventually get to a point where it's extremely useful and could change up certain industries (game assets, marketing materials etc).
As a new user of copilot for the last three months, I can't disagree more. I was initially skeptical, and I have noticed it often produces code that looks good but is wrong. However, it still saves me enough time each day to pay for the monthly subscription - in one day. That's a 30x ROI. I imagine it only gets better from here. I wont go back to programming without it.
Can I ask what kind of programming you do for it to be so helpful?
I mostly do maintenance of legacy codebases (also known as codebases, lol) where a lot of the work is figuring out where the changes need to be made and actually making the changes is frequently just a few lines here and there.
When I do have to figure out how to use some API, it's often not an open source one, so Copilot would not have it in its corpus.
I think these kinds of conditions are really common since software tends to last for maintenance longer than it is in initial greenfield development.
So I'm confused what kind of work benefits from Copilot. Just pumping out greenfield development of new websites/webapps that don't use much legacy or closed source code or services, just using existing popular open source libraries in commonplace ways?
The other thing I wonder about is code quality. When I look up API docs and stackoverflow examples, I get to read them all, maybe test some examples out in a CLI/REPL, and then decide carefully exactly what to do myself, what special cases to worry about or not, what errors to handle, etc.
Maybe what I end up writing is even the same as Copilot would have written. But in the process, I learn about finer details of the library and make detailed decisions about how to deal with rare edge cases. Might even end up writing a comment calling out a common pitfall I realized might exist.
My question is -- in order to save so much time with Copilot, are you still able to do all this extra thinking and deciding and learning (in cases that warrant it)? Or would doing that just end up consuming most of the time Copilot "saved"?
In other words do you end up producing code much more rapidly, but at the expense of code that looks more like a junior than a senior wrote it, because it is most concerned with working at all, and does not have time to worry about finer details? At the expense of not being as deeply familiar with the foibles of the API you're working on?
Honest questions as I haven't tried Copilot, and these are the thoughts that make me imagine it won't be of value. A lot of what I know I learned from doing the parts of the work that Copilot would be automating. Sure, Copilot would save me time when initially writing it. But would I then have less deep knowledge available when there's a fire in production because I never explored the fine details of my dependencies as much?
I haven't tried copilot either, but one of the things I'd be curious about is how well it can conform to a company's coding style guidelines and/or match its coding style with the existing legacy code that's being modified.
One of the major annoyances of working as a team with legacy code is when someone forgets to, or deliberately avoids, conforming their code to the style and techniques of the surrounding code. Nothing grinds my gears like working in a 500 line C++ file, where_every_function_uses_underbars, has consistent 4 space indentions, avoids exceptions, and passes by reference, but right in the middle is that functionThatMiltonWrote that uses camel-case, has 8 space indentions, throws exceptions, and passes-by-pointer.
I haven't tried copilot in languages less opinionated about style, so I'm not sure how well it handles that case. In Python, TypeScript, and rust it seems to work well. In any case I use auto format on save, so that fixes some of the potential formatting issues.
It works great in our codebase; it uses the current text in the file from above your cursor as reference.
So if you're creating a new file, it isn't always perfect, but once it catches on to your style it's seamless.
It's really more like a smarter autocomplete. I haven't tried it on a third party API yet, we don't use many at work. I work in a startup on a Python and TypeScript code base. To give an example, last night I was creating a unit test and copilot filled in the assertions. It missed one it couldn't know about, and it got two wrong. But it was a lot faster. The most amazing case to me was with a function to transform URLs to an image resize service. There was a bug in the function, it needed to return URLs ending in .SVG as-is. I went to fix the bug by typing "if" and copilot filled in the `if url.lower().endswith(".svg") return url`. It knew about the bug before I did. Too bad it couldn't do a code review when I originally wrote the function.
They have a 60 day free trial. Try it out, it's one of the most interesting changes in developer tools in a while. I feel like I'm living in the future sometimes when using it.
Whoa! Really? Seems like modern software releases would be excited to achieve 98% with the incredible amount of bug fixes/patches released very quickly after the massive beta test known as release day.
What's your metric for the "percentage the code is wrong"? Is it how many lines of code were wrong, or how many test cases the code fails?
Presumably if AI-generated code passes every test case, but would fail on edge cases that some human programmer(s) did not anticipate in their suite of tests, the humans potentially might have made similar coding mistakes as the AI if they had had to personally write the code.
Code has to be 100% correct or else it’s considered a bug (assuming it’s syntactically valid).
Code that is 98% correct is actually much worse than no code at all. That’s the kind of code that will introduce subtle, systemic faults, and years later result in catastrophic failure when the company realizes they’ve been calculating millions of payments without sales tax, or a clever hacker discovers they can get escalated permissions by passing a well-crafted request etc.
It seems like you haven’t used CoPilot. Yeah, some of the harder bits I may have to code myself but the amount of boilerplate it reduces is incredibly liberating.
I agree, this would be an incredible tool. I can see how some of the outputs may help me improve a piece I'm working on, even if I would never use the model's output for my final product.
It looks like a decent way to produce concept sketches.
Currently, working with creatives (including programmers) is a very iterative process for non-creatives.
"I want this."
"No, I meant this."
"Can we try making that line longer?"
"Eh, I'm not feeling it. Why don't we try brighter colors?"
"Ugh. That looks obnoxious. Can we tone down the red?"
etc.
If the non-creative can use this system to reduce that iteration, it will make life easier for them (maybe not for the creative, if they are banking on the hours doing the iterations).
honestly i feel the opposite. one of the worst situations to run into as a designer is a client who is simply too married to a bad design. i would much rather work with something you scribbled on the back of a napkin than than some highly rendered ai vomit that is just fundamentally shit.
Does SD have the ability to take this kind of micromanagement input though? From what I've seen, it works off of a general descriptive prompt. Will adding a very specific "but put the duck 5 pixels to the left" or a very vague "give it more pop" to the prompt actually have the intended effect?
Not yet, but I can definitely imagine a future where these tools get more capable and refined, to the point where all the shortcomings listed above will be overcome. Knowledge about cameras and scene composition are already encoded in the networks to some degree, it just needs to become more accessible. There's probably also a better way to seed new images than by starting with random noise, so we could get similar variations easier. We have already made the big step towards creativity and real world understanding of objects and their lighting, the remaining issues are more technical and unless we are incredibly unlucky and run into a true show-stopper, we'll probably all have access to a high quality digital artist that can reduce production times dramatically.
You need to give some information about the scene to the network.
Camera settings is just a short hand to describe the field of view and depth of focus (at the very least). If you make that implicit you'd still need to give the network the steradians, focal length, circle of confusion, etc. etc. etc. that you want your image to use.
You'd need to understand everything in Hecht's Optics to tweak all the parameters of an AI generated image.
That's an implementation problem, not a technical or conceptual one. Diffusion models have shown that they can learn practically all of these things if you make them sufficiently big.
Of course not, it just gained a new "advanced stable diffusion usage" track. Photography didn't displace painting from art school, and even outside art there are plenty of examples to be found. CNC machines are awesome, but every machining class in existence still teaches manual lathing and milling.
Nobody gets paid to be a telephone operator or a cobbler or a steeplejack anymore either. If new tools come along that automates and further commoditizes the production of art, I don't see the issue. Automatic driving for trucking, public transport, taxis would economically impact vastly more people, and everyone here seems to be cheering for that.
There's nothing special about art. People will still do it as a hobby, like they do with a lot of other dead trades and professions. In many ways I think the personal aspect of the creation and sharing with people you know is the special thing, rather than the creation or performance of a commercial success (not that I've had any experience with the latter, mind you). And I'm not against the mass consumption of art, just that it needn't be produced by people if it has the same entertainment value then that's great.
Well you got me. I guess chimney sweeps and black smiths and loom weavers and carriage drivers still exist too, not sure about steeplejacks since the passing of Fred Dibnah. But you get what I'm trying to say, hopefully.
You included “cobbler” only because it sounds old-timey to you, and I was pointing out that it’s not.
Repairing expensive shoes is not an automated process. It’s more like fixing a roof leak, landscaping, or changing a flat tire. Jobs for those things still exist and aren’t going anywhere.
No I included it because it used to be a common profession and now it is an extremely niche one. I could have also put chimney sweep in there knowing that it's still a thing.
Someone is still going to pay for a person to perform or create art for them. Some professional driving jobs will continue to exist long after most are automated. When I said "nobody", that's what's known as hyperbole.
You're nitpicking one of the given examples without engaging the user on the point they were trying to make.
To be nuanced, maybe they might have said, "cobblers are less in demand now that many people have moved from owning fewer pairs of shoes they make last through repair to owning more pairs of shoes that they tend to get rid of when they are worn out due to changes in construction materials used in production," but if people have to write like that to make points, nobody will ever make a point.
It's a nitpick, but a little bigger than that. It's as bad as including "bus driver" in the list. Cobblers just shouldn't be included in the category at all.
Cobblers are in just as much demand in most of the world as they always have been. They only fell out of demand in car-dependent areas, which is a small minority of the world population (but a vast majority of the HN commenting population since most of the USA outside of a few cities is car dependent)
I don't know if it has anything to do with construction but doubt it. If you actually walk everywhere shoes don't last very long these days, especially shoes under $100.
There are a lot of other urban services that exist in almost the entire populated world, but that most Americans think quaint because they are not relevant to a car dependent highway world.
All that said, this really is a nitpick and the original point stands very well. Some of us just don't like it when car-dependent people forget that they are a small minority worldwide and instead treat urban walkable people as the insignificant minority! Or rather, HN being a forum for all things interesting, we find it interesting to make it a teachable moment. What could be more interesting than finding out that something that has always seemed obvious to you is actually backwards?
> I don't know if it has anything to do with construction but doubt it. If you actually walk everywhere shoes don't last very long these days, especially shoes under $100.
By construction, I mean the material and design of shoes people tend to wear. I can't say I've ever met someone who takes sneakers or running shoes to a cobbler and these shoes are more common nowadays.
Those kinds of shoes you mention tend not to last very long at all and are not resoleable. If you walk a lot, you find yourself throwing away the $80 shoe after just 6 months.
These kinds of shoes are most of the market because most people don't walk much in the USA. If you walk a lot, you might still not change anything and keep buying the disposable sneakers, throwing away $160 a year.
But if you walk a lot AND are disposed to think critically about the situation, you find that if you pay a bit more for shoes you can make them last many years as long as you resole them periodically. And as I recall, a good $50 sole on a good $150 shoe costs half as much and lasts three times as long as a disposable $80 sneaker.
Not only do you save money (not really a ton) but it actually is more convenient, since even counting resolings, you get more miles between having to go repair or replace your shoe. And you don't have to wear in your leather uppers again. It is truly a luxurious feeling when you come back from the cobbler and have shoes that are worn in and fit your foot just perfectly like a glove... yet the soles are brand new and strong and comfortable and ready for another thousand miles.
Why do you think there's a stereotype of leather boots being popular in NYC? I'm sure the resoleability and longevity in the face of large amounts of daily walking have a lot to do with it.
I grew up in a car dependent area and moved to a walkable big city in my teenage years, and only then became aware that cobblers were still a thing -- in fact, in NYC, they're not an obscure thing, but as common as bank branches.
I think that's why there's a lot of people online who think cobblers aren't a thing anymore. They're from car dependent areas. If you drive everywhere, shoe soles don't wear out much faster than shoe uppers anyway, so it doesn't make sense to care. But if you move to a walkable city, you'll suddenly find it quite economical, since the soles wear out far faster, and the cost of a sole replacement is less than a new pair of shoes, so you might replace the sole a couple times before discarding the shoe.
i dont think it would be a massive stretch of the imagination to be able to train a robotic arm to hold a brush and reproduce something that is on screen and do it really well.
Machines like that already exist but specifically with oil painting there is a lot of important information in how was the paint applied. So it is really really hard. Too many decisions in tools used, angles of the strokes, pressure, thickness of the color etc.
You might make painting that looks identical from far away but when you come close it will be completely different.
No, it didn't but people who took only one semester of art classes may remain under the delusion that art is about making pretty pictures.
People who need illustration or graphics with no particular style can meet their needs with this tool but that is far from art. This replaces commercial illustration, not artists.
Exactly, this replaces general illustrative commercial art. This is great for illustrations at the head intro of fiction stories. However, fine artists that produce art where the object created is merely the totem for their actual art: intellectual conceptualization of an idea, often concerning the finality of life and the immature behaviors we playout within; that type of Fine Art is completely out of reach of modern AIs. I'm sure there are attempts by AI developers to mimic such art, but that mimicking will be gibberish. It requires a sentient being to create Fine Art, because Fine Art is an 360 degree expression of existence, not some pretty picture.
I can see this replacing the clip and misappropriated art in slide presentations that no one is paying for currently with a $20/month service that lets you do "line art angry man in toga at computer" to put into your talk on Kuberentes.
It might replace the "nice pic, do you have it at {size} so I can use it as a desktop image" comments on various social media sites.
I don't see it replacing an actual photograph as a wall hanging (or a painting) because they are subtly wrong in some ways. The reflection in the lake doesn't match the landscape... that one cloud has its lighting at slightly different angle than the rest.
Possibly... but I'm skeptical. For the type of photography that I do, these are things that come from an understanding of the world and its implications. You aren't going to see certain types of clouds in certain landscapes - they just don't form there. For example, having a cloud that indicates fast moving wind in a mountain environment in a plains landscape with a glassy smooth pond.
It's not that you can't paint that picture... but you'll never be able to capture that scene in camera. If someone was presenting that as a photograph, it would feel wrong to me because of an understanding of the meteorological criteria for the scene.
Yet, I am doubtful that I'll have a generated image at 11x17 that holds up to the same scrutiny that I apply to my own photographs.
I am absolutely certain that it will be able to generate images that are completely appropriate for images that you don't look at for more than a minute at a time or are used as complimentary material for other content.
All that said, I am not concerned that I will get more or less sales of my photographs with AI generated art competing. The people who are going to pay for a photograph are going to pay for a photograph. Those who aren't - weren't going to in the first place.
Yes, but the consequence for society is that professionalization is coming to every single field, and the blue-collar jobs are evaporating, and we had better figure out how to get all those people either white-collar jobs or some other form of income if we want to avoid significant social instability.
It used to be that you could find work as a political cartoonist and draw a picture of some satirizable politician each day - maybe Dr. Oz with a funny-looking vegetable platter, or Biden falling off a bike labeled "build back better," or something - and that would be a career. Now it seems you can just type "Dr. Oz holding a vegetable platter, political cartoon" into one of these AIs and the bulk of the work is automated. Sure, you could spend the rest of the day refining it, but nobody's really looking for perfection or transcendent skill in their political cartoons.
You can still find work making political videos of Dr. Oz and vegetables (e.g. https://twitter.com/JohnFetterman/status/1564432981841907713). Today's image generation AIs cannot do videos like that. Tomorrow's will, before we know it, and again nobody's looking for more than a baseline level of quality there either.
And even the local newspaper that might have been employing a political cartoonist is being swallowed by high-capital-ownership companies that can replace a lot of the writing with AIs. The expectations of quality are a bit higher - though the AIs have mostly gotten the hang of sports writing - but again people don't expect The Smalltown Gazette to have the standards of The New Yorker.
Sure, the highest-quality drawings, feature films, and longform journalism will get a lot better. And that's great! But most people don't work on such things. What are they going to do? If nothing else, how will they remain an audience with money to spend on the highest-quality works?
(I am not advocating for stopping this process, to be clear. Smashing the AIs isn't a coherent proposal, and smashing the physical machines of automation didn't actually work when the Luddites tried. I'm advocating for admitting that this process is happening and drying up employment, admitting that keeping as much of humanity as possible under a good standard of living is important for humanity, and figuring out what to do about it.)
Is it possible to provide another image as a prompt for SD? For example, could you provide a simple drawing of a house and expect it to render a house?
You can do using img2img method based on top of SD. Another option is to create an embedding of a concept based on textual inversion and then use the embedding to guide SD generation. Both methods are possible using this: https://github.com/hlky/stable-diffusion-webui
Thank you for this! I've been playing with the optimizedSD for a while but couldn't get what I wanted out of it. This guide makes sense to me. Gonna give it another shot on the weekend!
I think this can be seen a bit like the invention of an "index fund" for art. Active investors are still needed to generate the market signals that an ETF can aggregate, but for the majority of people an ETF is preferable to the cost of hiring an active investor yourself. And similarly SD needs artists to generate the signal that it aggregates, but for the majority of people it might be (or might soon be) preferable to use SD to get a "generic"/"average" result instead of hiring an artist yourself.
There is bound to be a smart kid who already turned this idea into a shitcoin (meaning a pump and dumb money grab, not an actual attempt to make a art-index and tokenize it).
It seems someone indeed did this (the p&d, not the index): https://www.coingecko.com/en/coins/artonline
Ultimately software engineering isn’t about code, it’s about describing in sufficient detail exactly what you want. Of course, this just leads to business analysts and ux designers being the next ones to be replaced after coders… :P
> A photographer who intimately knows the different lenses and cameras and industry terms will get to a representation of their idea much faster than someone without that experience
There are already websites which sells tailored "professional" prompts for dall-e, GPT3, etc
I always wondered if we massively overestimate human creativity. Maybe it is ingrained in our culture and our very being. I’ve never heard counter arguments that humans are not that creative.
Creativity demonstrated by Alpha zero chess engine blows Magnus Carlsen’s mind (from his recent interview with Lex Fridman), I wonder if at some point in the future, we’ll finally throw in the towel and get out of the denial phase.
I believe the creativity is primarily the process, something we enjoy doing, not the method to _produce_ or _generate_ something. Sort of like dogs like playing fetch. Some creative processes do not create permanent artifacts at all, like all performing arts.
Without trained models from human creativity, what can AI do ?
These AI emerged because of human creativity.
Picasso created it’s new art form from its own creativity. He created something no one ever though of before. Now AI are fueled with Picasso’s drawing and can produce art that looks like his maybe.
But what about creating something entirely new that has never been fueled into the engine.
Could the AI invent something I’m about to dream tomorrow and this be the exact same copy ?
I think it's undeniable that AI can create novel things, the question is if AI can create novel things that are also interesting. A randomized 600x600 png is novel, but it isn't at all interesting, much of what goes into making a piece of art interesting is not a quantifiable or well-defined goal. That's not to say that AI is better than humans, just the opposite, art is a deeply human object, and I do not know if we would appreciate art made and developed by AI in the same way we do absorbing and creating art in response to each other.
I see your point.
We agree that on a 600x600png there is a finite number of possibilities given a set number of colors.
It could be possible to « brute force » all the images possible which would be the equivalent of trying all the combinations of letters to write a book. So creating things is not really a problem. Creating relevant things from a purpose is.
The fact is, we know no other intelligence except ours. We are the only known being that appreciate art or books, from our subjective conscience.
Can AI create its own art, understood and appreciated by itself ?
Has intelligence a meaning without « humans » ? It’s part of the same thing. So the AI is modeled after ours. So it’s not its own thing and cannot understand what it creates and if what is created is relevant. AI is just an amazingly powerful tool.
No, apparently we just massively underestimate how important studying the humanities is.
This is illustration, not art. I'm flabbergasted how many people are so quick to confuse or conflate the two.
For the record, my significant other is an artist and my entire life has been surrounded by creative people. I’m questioning human creativity, say, 50 years from now, and how much machine creativity we currently underestimate. Sure these artworks lack imagination and fidelity today. Appreciating art and humanities has nothing to do with projecting AI capabilities in the future, I’m flabbergasted how many people are so quick to confuse or conflate the two. Maybe we should require people to study logic/reasoning and how important it is.
50 years from now, I have little doubt that will be the case, barring serious civilizational decline. But that isn't the sentiment on this page.
Ditto for me regarding being surrounded by creative people. This isn't going to lose them any business any time soon, not even the younger ones five years from now.
Alpha zero was trained by playing against itself, with a clear win condition. You'd need to quantify what it means for one image to be "more creative" than another to get a similar result.
"AlphaZero would sacrifice a knight or sometimes two pawns, three pawns, you can see that it's looking for some sort of positional domination, but it's hard to understand. It was really fascinating to see. "
I've seen an inpainting technique where you put an existing character then describe what their twin is doing, then crop out the existing, which seems to persist the character at the cost of fewer pixels to work with
I've tried too, and honestly I'll hire artists again (I did several times). It's easy to come up with nonsense, so in the end I think this is a tool in the hands of artists more than anything else.
For threatening artists it should be much better. Maybe it's not the model, maybe it's the interface human-computer that fails, but in the end it's what we have.
Maybe someone with a lot of time in their hands can iterate enough times to come up with something nice and depicting what they wanted. Not for me.
That’s possible, but my gut feeling is that this turns out to the the same as the DARPA grand challenge moment for self driving cars.
It was an absolutely amazing accomplishment. I legitimately thought I could hold out long enough that my next car would be fully self driving. Truckers were an endangered species.
But here we are 20 years later, and we’re still almost there.
We’ve made amazing progress and I love the self driving features I do have on my car, but how many jobs have been replaced by self driving cars?
Well, it won't replace creativity, it just makes starting out easier.
SQL was designed so that business people could describe queries in English. How did that work out?
Now, of course, we have ORM's, which will get you data, sure, but often in egregiously inefficient ways if not used correctly. If you want to to get it right, you still need to pop the hood and adjust things, and you have to know what you are doing.
> A photographer who intimately knows the different lenses and cameras and industry terms will get to a representation of their idea much faster than someone without that experience.
Sure. Only now the gap has been shrunk by a factor of infinity, because the time to representation was infinite for someone without that experience before.
It'd have to get itself into the dataset, as if it was 'Snoop Dogg' or some other identifiable person that the AI can reproduce. The degree to which you can generate stuff that is deepfake 'Elon Musk' doing whatever you say, is the degree to which you can invent a character and have the AI generate images (or video) of it following (sort of) your script and directions.
I think there will likely be better ways to bias the training of specific instances - Something like a 'training library' with biased training data that you can plug in for your use case.
For example let's say you're a well known designer with a distinct style, and you train your own instance on your lifetime's body of work. Now you can generate whatever you want in 'your style' (just like you can now ask for a painting in Dali's style).
Now you've turned your style and design process into a factory - for every new client you can create whatever they want, in your style, with multiple examples, in a button press. Perhaps you can even sell that 'training library' to other designers?
I think its because we've eveolved to be more sensitive to faces and pay more attention to them. I suspect the whole image is off, but we really notice it in faces and particularly the eyes.
You are exaggerating. HN is fairly broad in it's politics and there's often quite challenging discussions around topics related to diversity. Whilst there is a tendency towards "Silicon Valley neo rationalist" and other voices are sometimes in the minority, you using a term like "suppressed" comes across as slightly overdramatic.
> you using a term like "suppressed" comes across as slightly overdramatic.
I don't think so, because the person I was responding to was flagged. Abusing the flagging functionality to silence difficult opinions about race is not uncommon on this website
Probably because:
1. You don't make any actual point that could lead to a constructive discussion.
2. What you are saying is not true: this model was developed in an university in Germany and sponsored by a private company. The only public founder I could find is this one: https://www.crunchbase.com/person/emad-mostaque
Perhaps you should reevaluate your own biases as well.
> we grabbed the data for over 12 million images used to train Stable Diffusion, and used his Datasette project to make a data browser for you to explore and search it yourself.
> Read on to learn about how this dataset was collected, the websites it most frequently pulled images from, and the artists, famous faces, and fictional characters most frequently found in the data.
LAION collected the images used to train Stable Diffusion:
> All of LAION’s image datasets are built off of Common Crawl, a nonprofit that scrapes billions of webpages monthly and releases them as massive datasets. LAION collected all HTML image tags that had alt-text attributes, classified the resulting 5 billion image-pairs based on their language, and then filtered the results into separate datasets using their resolution, a predicted likelihood of having a watermark, and their predicted “aesthetic” score (i.e. subjective visual quality).
One day, images generated with these AIs will become ubiquitous online, and new AIs training on data will wind up with feedback problems because they got trained on photos that are AI generated, possibly even ones they themselves generated!
This tech is a huge deal. Huge. Ubiquitous unique, beautiful art. Entire movies made by machines. Too many to even watch them all. Actor models that aren't real people owned by the thousands by people and corporation s.
or when you try to google something that would better be answered by a professional that costs $1000/hr like doctor or lawyer but you just want a vague idea of the landscape and all the answers are "you need a lawyer/doctor" and not any real attempts at an answer. just shutup!
The images I generate don't have a watermark. Same applies for a lot of other AI generated images. The watermark is the most trivial part to remove and is definitely not a defining feature of what makes an image AI generated.
It reminds me a bit of "Low-background steel", which recently popped up again here. Images older than 2020 are predominantly human generated. Anything later is suspect!
Expect hit actors from today to become digitized and immortalized on the screen. Eventually there will be a cannon of standard "actors" (customized per movie of course) that will never change once they settle in on the appropriate archetypes.
This already happens. Around 2010 the actor for the colonel from Avatar had his face digitized for use in sequels. More recently actors from earlier eras have been reincarnated in CGI (Luke Skywalker takes on his original trilogy appearance for the Mandalorian).
Although it's likely that the images that appear online will be peoples' favourites of many candidates, also likely developed through many steps of AI tweaking. So that's still a useful signal for another AI to consume.
> One day, images generated with these AIs will become ubiquitous online, and new AIs training on data will wind up with feedback problems because they got trained on photos that are AI generated, possibly even ones they themselves generated!
Possibly but not necessarily. The human curation prior to posting as well as additional textual context associated with the image could be valuable training signal, even if there is some feedback.
These systems make for a good family game in our house. We take turns with prompts and have the laptop broadcasting to the TV. I imagine you could make a more formal party game from it - some sort of weird version of Pictionary perhaps?
I'm honestly going to think about game ideas and try some with the family. It's fun to play with these models on your own, but it's way more fun with a group of people laughing and one-upping.
(Also, very happy for somebody to steal this idea!)
Why do Stable Diffusion images all sorta look the same. They look either deformed or like pastels . I tried it a few times on some free sites, and it was not accurate. I would put in 5 keywords and maybe it would get 2 of them right. These are common search terms.
It's not a search engine. It takes a while to get a feel for what prompts work and how to phrase things. I suggest browsing https://lexica.art/ and using some prompts from there as a template.
I tried SD for a bit and the results are at best mediocre.
Dalle2 is dimensions better at understanding prompts, but it’s still a beta. So there is hope.
Linux started out pretty mediocre compared to closed source Unix, and even today there are operating systems that are better, but being freely available made it a game changer. I don’t foresee Dalle2 being open any time soon.
I'm struggling with the connection you're making between Linux and Stable Diffusion.
Linus didn't invent the core concepts of Unix. He copied the (arguably) good parts from an (arguably) closed ecosystem. His big innovation was leveraging the internet to create a new kind of community not really seen before it. The Linux bazaar gave smart developers excluded from the Bell Labs / BSD cathedral a place to be productive in that style on their own terms and others used it to disrupt the OS business. A lot of shit gets into Linux that the community turns into useful things.
I look at the inputs/outputs of Stable Diffusion and am reminded of the fractal craze of the 80's which revealed the underlying simplicity of incredibly complex looking things. Lots of interesting art and technology came out of that but I don't think anything like the Linux community did.
yeah one thing I think of is when an artist 'creates' something, its coming from their brain, like expressing what they feel. When AI generates the Art, even though there is a human curator, the lines and shapes the AI picks out is all random/arbitrary/no-emotion (even though it draws upon a huge dataset of lines/strokes/styles to form). It's "noise" art. The noise is just cleaned up to resemble real things.
They are very clean looking, it could be an effect of SD's compression to be so small.
MidJourney's production model is good about dirtying things up, you could use it as a post processor. I think its default style of turning everything into fine art-cyberpunk-movie matte painting gets old though.
I've gotten some good atmospheric photorealism results out of DALLE, but currently trying to get outpainting to work to extend them and it's tricky.
Yep it’s missing intent, just like machine translation. There somehow seems to be less information than there is in the input. I think more people will start to notice it as time goes by.
> There somehow seems to be less information than there is in the input.
This captures my thoughts very well. That's why these images get old very quickly - you can basically imagine the same thing in your mind. There's no real whimsy, surprise or creativity there. For now at least.
Having seen some of the pictures of AI art (I don't know if they use the same or different programs or models), my opinion is that the pictures are not good enough, although sometimes they are almost good enough. (I think there might still be a use for it, despite this. Some other comments mention some possible ideas, although there may also be others.)
It's rapidly getting good enough for things like stock images for blog posts. Where you just need an image vaguely related and don't want to pay for an actual stock image.
After the novelty wears off and everyone has already seen tons of these automatically generated images, will writers still want to adorn their blogs with these generic illustrations? Or will it turn negative: "If you have nothing to say, add an AI illustration."
I could see it becoming a marker of low-brow taste where vaguely related illustrations become something akin to decorating your home with velvet paintings of Elvis.
Prompt writing is a challenging and creative endeavor like any other, and a well-written prompt/image need not be mearly "vaguely related"; Certainly not in comparison to the stock images that are already utilized today.
While I agree that they usually look pretty bad (and that the resolution is too low), some of them have been on par of what I've seen in video games. Stellaris in particular comes to mind (my memory might not serve right, haven't played the game since release). So I think what we're already have is good enough for some use cases.
I am a contrarian by nature. Wearing my investor hat, I continue to be unimpressed with AI tools, including the latest image generation enhancements. I do find stable diffusion interesting, but I don't see the disruption. Similarly, I don't see the disruption from Github Copilot considering I run a company full of highly paid and extremely skilled developers and not one of them uses copilot.
How often have you wanted to hire a conceptual artist? I have never done so in my life. While I have visited many art museums and appreciated it, this has not affected my commercial endeavors.
I think if I was a creative writer or professional artist, the current generation of AI tools could be useful for inspiration. But the bar for quality is just so, so high in the creative industries, I am not concerned that AI will replace them.
It's September 2022 - where is my self driving car?
I can think of two good businesses you could build with SD right now. Neither of them are text to image models or necessarily "art", both involve using it to visualize other kinds of sensing by transforming them to image embeddings.
Moravec's paradox is actually an observation only relevant to 80's state of robotics and AI. I don't think it holds true, and low-level stuff does absolutely not cost more than high-level stuff. What holds full self driving back is the vast amount of rare edge cases that can't be dealt with by current methods. (largely because the system is only trained on available data and doesn't understand first principles)
> I think if I was a creative writer or professional artist, the current generation of AI tools could be useful for inspiration. But the bar for quality is just so, so high in the creative industries, I am not concerned that AI will replace them.
I don't think AI will replace professional artists, but instead supplement them. Creation of Photoshop, Wacom tablets, 3d modelling and animation tools didn't put professional artists out of their jobs, it gave them new tools to create even better stuff in less time, and I think this is what's happening here too.
What affect will it have on kids? Every kid can now just stick figure draw into painting or else ask Siri "draw me a picture of Trogdor holding up a sword" etc... I can imagine that having some kind of large influence on creativity (not sure negative or positive)
I think you're viewing this the wrong way entirely. Don't view AI as something unique, it's just a class of algorithms. View it in the same way that as the anti-lock brakes on your car. Did anti-lock braking systems entirely remove your need to operate the brakes? No. But its net effect was to massively reduce the number and severity of crashes.
All that these algorithms are going to do is slightly bump up the productivity of some people. But that's amazing. Productivity is basically the free money tool. If people are more productive, they get richer for doing less, and the single best thing we can do to improve people's lives is make them richer. Poor people? Make them richer and they'll be happier and healthier. Rich people? Make them richer and tax the hell out of them, make the world better for everyone. Productivity is the best tool we have for making some people richer without making others poorer.
I’m going to partially agree and partially disagree with you.
Agree that Stable Diffusion seems a bit over hyped. It is very cool! I am definitely going to play with it. I’m not sure really how much it’s going to change the world - there will be some neat apps and it may put a certain class of concept artist out of business (note I am not saying it will eliminate them - there will always always always be human artists creating wonderful art. But as a business model / way of earning a living, that might change).
I am going to disagree with you on GitHub Copilot. It has radically changed the way I do software engineering on my personal projects and it has increased the efficiency of the engineers at my company by at least 10%, maybe more. You should check it out. It eliminates an entire class of puzzle solving, and the least efficient kind (“how do I drop a column by condition in a pandas dataframe again?”). Simple answer via Google, but maybe 3-5 minutes of reading. 10 seconds via Copilot.
> No-one expected creative AIs to come for the artist jobs first, but here we are!
Maybe that's because we never really thought about it. In hindsight, it's only logical. For an artistic rendering, correctness doesn't matter much nor does understanding the model. For flying a plane or driving a car or just transforming code from one language to another, it very much does.
There are some kinds of “correctness” that matter. When you’re doing something new, you can only really do one new weird thing at once. Too many weird things and it falls apart or distracts the viewers.
(The “cow tools” Far Side comic is a dumb example.)
Since these AIs can’t count, that extra weird thing is probably going to be people with too many fingers. I actually have a personal collection of weird Midjourney images because if you ask for wide aspect ratios it starts generating multiples of the same thing and fusing them together…
That depends on how easy it is to use the new tools.
Getting higher level languages and various libraries/frameworks didn't make coders obsolete, because there was still tons of work to do at a "higher level" that required algorithmic/engineering-style thinking. Yes, previously difficult tasks got a lot easier, but that just meant that even more ambitious projects could be tackled, and that was useful.
In contrast, it may be the case with art that the things a relative newbie can spit out with the help of image generation models are "good enough" for most use cases. Yes, a real artist would be able to squeeze out even more, but many companies may be willing to go with the option that's enormously cheaper and still seems to produce okay results.
Is the same not true for programming? There's a lot of non-programmers that create programs in Excel, or use no code solutions. A relative newbie can still get a Wordpress website up and running, you only need professional SWEs for the heavy duty stuff.
For better or worse, you very very quickly pile up complexity when programming in a way that novices can't deal with (and experts too, often).
What we're going to find out is whether novice users of image generator AIs frequently get stuck with inadequate results. IMHO the answer is likely to be "no", especially after we've iterated on the tools for a few more years.
Agreed, but the rub is that there was a ton of demand for "heavy duty stuff"; indeed, the improvement in code tooling actually increased that demand, because software programs became so much more powerful.
It's not clear that you'll see something similar here, that easier-to-produce high complexity art will greatly increase demand for said art.
I don't know about that. A lot of the images I've seen SD generate have been interesting but "incorrect" in really important ways that indicate it doesn't _understand_. Things like people's limbs not being connected correctly, or three legs, or a wheel on a car not being in the right place, or elements of perspective that are just off. All things a real artist would never do unless it was intentional.
It's like some of the stuff that GPT-3 generates, it sounds extrememly plausible and realistic but then you read it properly and realise that parts just don't make any sense.
However I don't think these things are show-stoppers, I think we'll see a lot of artists getting good at providing the "feedstock" and guidance to these systems to deal with those quirks and generating some really interesing work.
Symbolic AI dominated the field for 50 years: thanks partially to an accident of history (the highly influential work of Minsky, Newell, etc) and partially due to lack of the data needed to try anything else.
Now we’ve seen essentially 50 years of research on data-driven methods compressed into 5. In retrospect it makes sense that applications to which symbolic reasoning is especially ill-suited (including artistic rendering) would be the lowest-hanging fruit.
Stabile diffusion is most of all a waste of resources! 20 billion parameters trained, using I dont know how many CPU hours, all for making better artistic images than what an amateur would do and on par with professionals. Art is nice, but not important for survival, contrary to smartly using the resources of the planet. Do you want to see a future way to live on or "the rest of Mona Lisa"?
I’m more trying to see what the utility of stable diffusion (or just the text to image problem) in the long term. Right now people can play around with making weird art pieces and maybe it will be integrated into design tools...but then what?
Eg with other AI problems out there I can see a potential application to medicine, self driving cars etc, but I just don’t see what the bigger goal of this is going to be.
Magazine illustrator, concept artist, stock photography, video game art asset creation, logo generation, interior design, fashion design, costume design, political cartoonist, caricature artist, website layout templates, font design, pixel art, book covers, children’s toy plush design, new car concept prototype…?
That's just existing industries. All the new crazy stuff it'll unlock... Custom operating system interfaces for everything based on the data they're working with, truly-open infinite video games where you can run or zoom any direction and have it write itself, bespoke lcd wallpaper customized on the fly to the person walking by, bespoke industrial design creating 3d objects (first digital, then printed) for exactly the context they're needed for merely materials cost...
I'm not really sure I know how to use these tools. I tried the following prompt:
"elon musk giving donald trump a massage using pizza sauce instead of oil, in a majestic room filled with flowers and golden toilets"
...and the result was a crappy AI-generated picture of not-quite donald trump holding a terrible rendering of a pizza, and some hands sticking out of random places. There were some red flowers in the picture at least.
I realize this isn't a "serious" use case, but clearly the tech isn't doing what I'm hoping it does.
I tried "coffee beans with cartoon mouths" and it's just a picture of some coffee beans. I don't get it.
yeah i'm getting the same. pick something much simpler like:
"A cup on a plate".
And then replace cup for other objects. It generates nonsense pretty quickly. It seems most able to generate stuff close to something which already exists in a complete form. Sort of a pastiche machine.
That's the absense of reason sticking out: the "ai" mimics, but doesnt understand, so its creations are frankenstein-like bodies, almost human and for this reason more revulsing. So this "ai" represents an insane painter.
I misspoke. I meant to say photorealistic faces. I haven't seen a good one yet from DALL-E 2 or Stable Diffusion. The systems that are custom built just for faces do a good job at photorealistic faces, though.
> [...] It takes a while to get a feel for what prompts work and how to phrase things. I suggest browsing https://lexica.art/ and using some prompts from there as a template.
I've been trying to get it to draw a picture of a man trapped inside of a light bulb. Can anyone think of a prompt that works? It draws all sorts of freaky things featuring men and light bulbs but none with the former inside the latter.
Try "a green field with sheep on it" versus "a green field with robots on it". From a human position that feels like it should be a simple juxtaposition of objects onto a background. Clearly not so for this model.
in this instance i didn't ask for an example. I was just observing that juxtaposing objects in a sentence doesnt work at all as you might reasonably expect.
Regarding the ethics. Thought experiment: I look at a copyrighted image, get inspiration, and then draw a new image (which is not the same as the copyrighted image). Is that ethical?
But if I do this millions of times, after looking at millions of copyrighted images, is it still ethical (as long as I don't recreated copyrighted images)? Tough questions!
Depending on how close it is it may not even be legal. To give a music analogy: at least in my country, if you take a melody, change all the notes' durations, and transpose it you'd still infringe
The ethics partly depends on the analogy you choose.
I can write a book about a boy wizard's adventures wizard school and that's legal, but if I call them Harry Potter it isn't.
I can create Harry Potter fanart and distribute it online pretty freely - but slap it on a mug and sell it, and that's illegal.
I can record an audio description of a painting that's as detailed as I like and it's legal to distribute - but take a photograph of the same painting and it's a derivative work, no matter how artistic my choice of camera settings.
We don't really have any prior examples that are precisely like these huge ML models trained on copyrighted data - and depending on which imprecise analogy you choose, you can come to a different conclusion.
Harry Potter is a trademark. It protects and distinguishes identity and authority. It is much harder to get a trademark, and is also harder to fairly use one. Different issue.
If your fan art is infringing, it was infringing whether or not it was on a mug or on dropbox.
Photograph one is not true. For commentary it can be by audio, or printed on a mug or whatever, commentary is transformative. Go out and take a photo of the world outside, if you live in a city you've captured thousands of copyrighted materials in your image. Maybe it captures someones painting, maybe it doesn't. Whether its fair or not is if it's transformative, the format doesn't matter.
I'm not seeing an ethical difference from any of this. Or did I miss the point?
> Harry Potter is a trademark. It protects and distinguishes identity and authority.
Perhaps I should have said Darth Vader, then - the point is you can copyright a character independent of the copyright on a book's text, and the trademark on the series name, and the fact that broad concepts like "black-clad masked evil overlord" are uncopyrightable. And that copyright can persist even if you transform a book character into an engraved coffee mug.
> I'm not seeing an ethical difference from any of this. Or did I miss the point?
The difference is:
If a person says "Stable diffusion is to its copyrighted training data as an audio description is to a painting" or "Stable diffusion is to its copyrighted training data as the concept of boy wizards is to harry potter" they would probably say it's ethically fine.
If a person says "Stable diffusion is to its copyrighted training data as a photograph of a painting is to the painting" or "Stable diffusion is to its copyrighted training data as video lecture is to a single image in its slides" they might well say it's not ethical.
Is your third example correct? Andy Warhol was famous for using photographs of existing art and transforming them only slightly. Roy Lichenstein literally took panels from DC comic books and magnified them and became celebrated in the art world for it.
In my opinion it’s just a technical side of ethics, which has much less value than the main side: what it does to an author. If they suffer from these copies (morally, financially, spiritually, etc) without a way to offset it, then it is not ethical.
No need to bring AI into it. What if someone chooses to open a Wendy's in a town that already has a McDonalds. The competition will defintely reduce the McDonalds's business. And many would argue (famously Rockefeller) that competing with another business isn't ethical when businesses could band together into trusts or similar structures to limit competition.
Well, but that's hardly a universal rule either. If we have a magic cure for bad eyesight, that'll be pretty terrible for optometrists, lensmakers and so on, but nobody would think it unethical that we're taking away their jobs.
To me it’s a different situation, because no previous hard, unique, “ownership” work of these people was used without consent to make this cure.
Think of this instead: a doctor collects a big volume of symptoms and analyses and creates a statistical way to cure people more easily. They publish many examples of their work without licensing anyone (legally and morally) to use it freely. Now some algorithm collects their data and many others data and transforms it into a better method. A doctor suffers from going out of business. Is that ethical? On one hand, the algorithm invented something new and easier to access. On the other, it basically stole parts of their and similar researches on a previously unthinkable scale. We humans copy ideas all the time and this is somewhat normal, but this enormous at-scale capability was never a thing.
Personally I don’t care for optometrists, uber drivers or designers. Nature will find a way. But when we talk about fundamental social contracts like property or accumulated knowledge protection, I think it is unethical to break them, regardless of technicalities. If it’s such a great advancement benefiting everyone, why can’t AI creators just ask permission for 2.3B of datapoints they used?
But that's what makes this so tough. Whether you consider the ways in which these AI models repurpose existing artistic works to be mere technical details or central to decide on the ethics of the matter depends largely on the analogies you reason by, as michaelt noted.
These new models completely changed my mind about how much impact AI will have in my lifetime. They are the most impressive software achievements in decades and anyone who has a “meh” reaction will absolutely end up looking silly.
I’m not optimistic that their impact will be positive though.
It is capable of surprising beauty. It will however be as beautiful as the disaster that is modern Facebook -- and the misinformation will be more convincing and subtler.
I think that the cause of that is unlikely to be Stable Diffusion but the lack of education that allows so many educated people in this thread to mistake images for art. In that way a lot of the damage has already been done.
This is happening with images first, but I can see how this might be able to be leveraged for all kinds of things. The speed at which society might be transformed could be astounding. But it won't be distributed evenly, not everyone is prepared to take full advantage of these benefits. This is early 2000's internet distruption x2 in half the time.
BUT it will likely be disrupting the upper middle class more than the lower middle classes, as it's mostly disrupting to creative work. Upper middle class people have a louder voice.
> They are the most impressive software achievements in decades and anyone who has a “meh” reaction will absolutely end up looking silly.
With the risk of looking silly, I declare "meh" once more, just as I "meh"-eh when GPT-3 came out.
The so-called "AI" is not fundamentally different from the AI of the 80s, it's just that now we have much better hardware. The main problem of the past AI winters still remains - the existing algorithms focus on statistical methods, which can be rather inexact. Imagine a nuclear plant controlled by an AI, or an airplane flown by a neural network. They completely lack reasoning capability and therefore you can't trust they will be able to adapt to unpredictable situations.
incremental improvements are easy to dismiss but it can be hard to tell when some critical mass of utility is reached. The first cars were electric (and steam-powered) but it took 100 years to supplant ICE (steam is next /s). the components for drone technology aren't new but control systems and batteries needed time to improve, VR has incrementally improved since the 80's, even the internet was around for 20 years before incremental improvement brought us the web...
It sure looks to me like this new development is hailed as a leap, rather than incremental improvement. Although I'm judging just by reading the comments in the HN bubble.
Either way, I think the current "AI" is fundamentally limited by the statistical approach to problem solving. Without any reasoning capabilities, no amount of incremental improvements will change the fact that neural networks are simply making guesses based on existing data sets. Nothing magical or revolutionary, it's the same thing we've known for many decades.
I will up you by saying that even your prediction is silly, for using the word "lifetime".
Not lifetime. Right now. Just one week of StableDiffusion in the wild has seen the development of many clients, GUIs, optimizations, plugins for commercial software, editing workflows, the list is endless.
The speed of it all is dazzling. It will not take long before this is assembled into a smooth experience that runs everywhere, with obvious end state being your phone.
And that's still just artistic image generation. The open sourcing of it means people can make any vertical, precisely optimized for a particular (commercial) domain.
Almost all shortcomings seem to be crushed in no time. Bad at drawing eyes? Here's a new encoder that fixes it.
The creator of StableDiffusion indicated to soon tackle music generation and if I remember correctly, even poetry. And there's 3D and video.
The ultimate end state: if you can imagine it, it can generate it. Not only are we much closer to that point than society realizes, we're also moving at an exponential speed towards it.
487 comments
[ 3.3 ms ] story [ 385 ms ] threadThe problem here isn't that the model was trained on copyrighted works, the problem is copyright itself and a cultural focus on collecting rent. We probably need UBI or equivalent to deal with the outcomes here in a healthy way.
Am I missing something here?
It's also quite obvious just by looking at the generated images that it's clearly transformative. The images generated are unique and you can't trace the original copyrighted image from what's generated.
You really don't need a judge to see that Fair Use covers Stable Diffusion.
Federal lawsuits are not cheap and the default is that you pay your own costs, you have to win the argument that you should get court costs & attorney's fees.
Or for written works, start with a sentance from a copyrighted work, or part of licensed code. Will it start reproducing that work word for word (like code pilot can do with the GPL license)? Getting these to generate copies of GPL'd, company owned, or other code with restrictions can lead to complex issues for the person/company using that code. Or likewise if a story contains significant elements of copyrighted works; worse if the works have trademarked elements.
Explain how you are not a copyright violating machine.
That might not always be true. I've gotten some results back that had the Getty watermark on them and others with the artist's signature. Unless the AI is adding that to images that never had one before (which might be a trademark issue), then you might be able to determine the provenance of the image components.
Is there something equivalent to the yellow dots printers add to their output that would survive the AI transformation?
If you take a step back, you can see that there are different ways to frame what is happening. One frame is: “Defendant built an algorithm that memorized features of Plaintiff’s IP. Defendant’s algorithm recombines parts of those features in order to produce works in the same domain that compete with Plaintiff’s work, all without Plaintiff’s consent.”
Bear in mind that copyright holders are among the most litigious out there. If generative art becomes as big a deal as some people expect, they will have every incentive to use their huge litigation budgets to claim a piece of the action.
Because the law develops very slowly, the legal process has not yet had the occasion to really evaluate what transformative use means in this novel context. I’m personally interested in seeing where things go, but it’s going to be a while before we know where the law is headed.
The fun part is, this is how human artists learn too.
Will the same image be legal if a human made it but not if it was created by Stable Diffusion? How will someone even know, short of a legal discovery process?
The model is open source and free for people to make their own modifications. I don't see how any watermark can survive in those conditions.
All this is super, super interesting to me. I don't think anybody really knows how it will play out, there's a lot of good arguments floating around.
I don't think computer-generated works will be easily distinguishable from human unless they're desired to be or shipped with metadata. It's already hard enough to distinguish human artists from other human artists without having names attached up front.
Yes, and tracing counts as art fraud.
Collage is a bit different, because you are mixing many clones of many other objects such that you create a new object; additionally the way you assemble the clones may transform them (a photo of the mona lisa has different surface texture than a painted version, even more different if it is clipped from newsprint), but while the borders of this are not clear, it is clear when people are far enough over the border. Think of hip-hop, sampling, and remixing music, and some of the legal battles which have come out of that.
I just spit out my coffee, turns out I'm not a bad artist, I'm actually a fraudster!
Yeah I am sure a lot of lawyers are going to have a lot of fun arguing every way imaginable.
Deep learning is very powerful and impressive in its applications to date. However, it’s so saturated with hype (and humans are so prone to anthropomorphizing things) that it’s often viewed as something much more profound than it actually is. Neural networks, despite their name, don’t model the brain. And they lack a whole array of “intelligence” features that humans possess and use constantly.
All of this is to say that there are very significant differences between computer algorithms and human cognition, and I tend to think the legal system will be unpersuaded by arguments that ignore those differences.
Also, this is to say nothing of the public policy interests that shape the law. Regardless of what’s “under the hood,” the law can simply treat human and machine output differently. I’m not a copyright lawyer, of course, so I can’t speak to the norms or technicalities of copyright law itself.
There are a lot of things we don’t know, but it is not magic. There is no discussion that artificial neurones don’t have much in common with the real ones, which are very non-linear and much more connected. But in the end it’s all electrochemistry.
> Neural networks, despite their name, don’t model the brain.
But that’s not directly related to my point. My point that even in the case of a ML model, you cannot get an exact reproduction any more than you can get from a human’s memory. In one case it’s scrambled somewhere in someone’s brain, in the other on a hard drive but the difference is not really relevant. Subjecting an AI’s production to the copyrights of all the things it’s been exposed to is very similar to subjecting a painter’s production to the copyrights of all the painting they have seen.
That’s an element of a belief system you may choose to subscribe to, not a fact. It can’t be a fact because there’s no way to prove or disprove it.
> “My point that even in the case of a ML model, you cannot get an exact reproduction any more than you can get from a human’s memory.”
Not quite. ML models can and do memorize and regurgitate near-exact features of the inputs. It’s not the goal, but it happens.
In the (actual) neurons, is there a representation of real numbers? Where are the numbers in the brain stored?
I feel like people who assert this neither understand what neural networks are and how brains work.
None of these details are relevant to the bigger picture similarities of non-hardcoded learning from training data. None of these details change the ethics of what's being discussed here.
> I feel like people who assert this neither understand what neural networks are and how brains work.
That's just your bias showing.
Don’t they, though? What else do you think is behind the curtain if not mathematics?
We don't actually know exactly how human artists learn, and human artists are capable of innovation, nobody knew pointillism or Bauhaus before they were invented.
A little know fact is that for humans it takes a long long time to learn, while they learn, they develop a style, if they don't they are not "real" artists, but merely executors, artists evolve, sometimes dramatically, in unexpected ways [1] [2].
So for us humans learning is an experience, not just recombining parts of features of other things.
We are also highly influenced by feelings, unfortunately, so sometimes we do things a certain way because we felt that way, not because we wanted to paint that thing that way, or because we are not good enough to do exactly what we wanted to do.
Is Mona Lisa happy? Who can tell?
Was Leonardo happy when he painted it?
What was Leonardo thinking when he painted it?
What was happening in his life?
Is that the best smile Leonardo could paint or it's an enigma he put there for future generations?
These questions are more important for an artist than the mere features of the painting.
The philosophical question is: is art discovered or invented?
If it's discovered, then SD can generate art, if it's invented, than SD it's not even generative work, because to invent something from something else, you need inventiveness.
[1] Picasso 1896 https://mymodernmet.com/wp/wp-content/uploads/2018/01/pablo-...
[2] Picasso 1946 https://www.photo.rmn.fr/CorexDoc/RMN/Media/TR1/MS4GY/16-515...
- It appears people can train AIs from scratch or at least fine-tune them at home.
- Even if your art isn’t in “the training set”, that does not prevent the AI from learning its style. (Someone can decode it to CLIP embeddings. It could have a really good text model trained on vivid art museum descriptions of your art.)
- The ability of an image model to generate your art means it could also be trained in reverse to recognize it, producing a caption model, which would give vision to the blind. And surely you’d feel bad about that.
You can also require cloud providers to enforce a ban on training (and deploying) such models, it's doable. Good luck training it in your basement, it will probably take you a decade.
If this is banned, it will become a lot like piracy - yes, it's available, no, most people (at least in the West) don't do it, practically no businesses do it.
Either use a CC0 set like Wikimedia/Flickr and throw in some dead artists like Brueghel, or train on data from a country we don’t respect the IP of. Lots of Taobao product photos out there. It’s enough.
As of about a week ago this tech runs on consumer GPUs. The weights have been downloaded 100s of thousands of times, and fine-tuning / modifying is possible.
Training from scratch is about $500k still, but it will only get cheaper and easier.
But I would not be surprised if this was trainable on a commercial GPU at home within that time. But I think another important trend that we are seeing is that you don't need to train these models from scratch.
Open-source "foundation models" means that you can usually get away with the much easier task of fine-tuning, as to not throw away / re-learn everything that these large models have already fit.
Edit: I initially said 2-5 years, but on more reflection this does seem optimistic (for training from scratch).
I don't know enough about diffusion models but if LLMs (of current size) have to use only public domain, they will be undertrained and we will see significant degradation in performance. Not to mention that Codex will be effectively dead.
Why would you, though? If art is for the sake of art, then all art is valuable regardless of origin. If art is for the sake of providing human employment, AI being better in no way stops performative make-work from existing. If art is for the sake of copyright trolls to troll harder, then fuck art, feed it to the AI!
Your "logic" for making AI art illegal is basically "don't like it". Your personal and subjective opinion is that it's not art by definition.. This is like refusing to eat artificially grown meat because you have some strange idea about what food "should" be. Even if the meat was made MORE delicious you would still claim it wasn't food and turn it away. There's no logical consistency to your position, it's purely reactionary.
What is not logically consistent is to claim that a black box utilizing statistical relationships between pixels in a giant dataset is an "artist" and that its products create "value".
The compiler is not a programmer, AI can never be an artist.
Step 1: Tweak settings and type text
Step 2: Look at the result
Step 3: If you like the result, go to step 4, otherwise go back to step 1.
Step 4: Save and share the result
Feels like art to me. Ultimately it's still a human using a tool to create art. For me AI art is just the name of the art style.
All this boils down to a simple fact that egos like to think of themselves (and of artistic interaction) much more than there actually is.
I remember a story when a literature teacher insisted on a definite symbolism of some minor detail in a novel. People contacted the author about it and he said no, there is nothing behind it. It was just a filler without any second thought. Makes you think how much symbolism is far-fetched in classics, where you cannot simply email an author.
If you tried learning, let's say, the chiaroscuro technique from Caravaggio you'd be analyzing the way the painter simulated volumetric space by using white and dark tones in place of natural lighting and shadows. You wouldn't even think of splitting the whole painting into puzzle size pieces while checking how many how those look similar when put close one another.
Given somewhat decent painting skills, you'd be able to steadily apply this technique for the rest of your life just by looking at a very small sample of Caravaggio's corpus.
On the other hand if you tried removing even just a single work from the original Stable Diffusion data set you used to generate your painting, it would be absolutely impossible to recreate a similar enough picture even by starting from the same prompt and seed values.
Given how smart some of the people working on this are, I'm starting to believe they're intentionally playing dumb to make sure nobody is going ask them to prove this during a copyright infringement case.
Both my parents (though retired now) were commercial artists. I was trying to be an artist at one point in my life before moving in Engineering and Science. All my parents friends are artists so I grew up around artists.
Ask any artist here who is using illustrator, Photoshop, Krita etc. How often do they google image search for textures, or reference images that gets incorporated into their artwork? The final artwork is their own but it may incorporate many elements from others artwork.
>If you tried learning, let's say, the chiaroscuro technique from Caravaggio.. You wouldn't even think of splitting the whole painting into puzzle size pieces while checking how many how those look similar when put close one another.
Ever seen hyperealistic pointillism?
Who are you to be the arbitrator of how an artists creates their work? Have you ever gone to a modern art gallery and seen all the different methods people use to create artwork?
Art is boundless and unique to each who creates it.
If a Artist uses a tool to create art, everyone agrees that is art. It could be a paint brush, clay, software on a computer etc etc. If an artist uses AI as a tool to create art then suddenly it's not art.
What potentially human-creatable images have I just taken ownership of?
Let's extend: what if I claim IP ownership of every image which StableDiffusion could produce?
For example, it has previously been litigated that nobody has IP ownership of an image taken with a camera by an animal.
Note: I am absolutely not a legal expert.
I don't think SD will replace most top artists for now. It's hard for me to believe SD is going to come up with images like those from top concept artists. But I can imagine SD replacing lots of situations, like maybe stock photography, when you can just ask the AI to draw "people in front of whiteboard discussing sales chart"
What's a similar thing that has come before this? I can't think of any, this is very novel. You'd want to wait for some rulings before you jump to conclusions.
As far as I understand it, this is still considered copyright infringement in most IP law systems. (If the samples aren't cleared)
So when your image generator keeps spitting out Gettyimages watermarks, while you are building a service that is in direct competition with Gettyimages for stock images, there is an argument to be made that Fair Use really doesn't apply here. As what you are doing is essentially stealing Gettyimages' work, AI laundering it and selling it back to their previous customers.
With StableDiffusion a Fair Use defense might have an easier time, as the results are released to the public. But it's still not exactly clear cut. If you type in "Mona Lisa", you'll still get something that looks like a copy of the Mona Lisa, not like an original work.
Everyone should be able to freely use any public works, and governments should ensure that people whose art is being used to train these models are able to continue to do so.
I first read this as a prediction that AIs will be employed to generate future IP legislation, and now I don't know if I'll be able to go back to sleep.
But two things I've noticed:
First, artists will still have a massive advantage over non-artists with this tool. A photographer who intimately knows the different lenses and cameras and industry terms will get to a representation of their idea much faster than someone without that experience. Without that depth of knowledge, someone might have to rely instead on random luck to create what's in their head. Art curators might be well-positioned here since having a wide breadth of knowledge and point of reference is their advantage.
Second, we need the ability to persist a design. If I create a character using SD, I need to be able to persist that character across different scenarios, poses, emotions, lighting, etc. Based on what I know about the methods SD/Midjourney/Dall-E are using, I'm not sure how easy this will be to implement, or if it's even possible at all. There will always be subtle differences and that's where being an artist who can use SD for inspiration instead of merely creation will retain their advantage over a non-artist.
That said, holy crap. This tech is insane.
See yesterday's Stable Diffusion article: https://news.ycombinator.com/item?id=32643564
I think that thing will neccesarily contained representations of dimension, behavior (physics/bones), and "style". Without the "ai thing", if only using an image/text, the character would have to be impossibly represented in the model, so it could guess all of these things predictably. For example, what that character looks like from a side profile, or behind. What if it's an alien, and its arms should always bend backwards? Could a text representation ever be made to completely describe this, with good reproducibility? Probably not. But, I assume some non-human representation would have a better chance.
As is, if something known is required, I think the behavior of these models can be considered "destructive" to the input image, more often than not. For this reason, I think artists are safe, for the time being. :)
[1]: Multi-View 3D Face Reconstruction with Deep Recurrent Neural Networks [2]: Deep Neural Network Augmentation: Generating Faces for Affect Analysis
You could use Stable Diffusion et al to create new characters based on a prompt, then farm the concept out to artists to produce individual works. Kind of like hiring a super expensive agency to design your new logo or brand identity, then using a stable of in-house designers to translate the concept into UI, ads, etc.
Any signals humans give (painting x is better than y) is another signal to encode. Take billions of such ratings and improve the AI’s taste to superhuman levels.
In short, anything that humans would add to artistically improve the outcome is just another signal to be encoded. It’s weird to write it, but artistic creativity is deciding what new pixels go where, which is a search problem (in a large search space) which AIs are apparently doing great at.
We have a bias: we’re humans, we must be important somehow! But it comes down to a bigger neural net eventually outperforming the one in our heads.
A neural net that can communicate with intent to human minds as well as a biological human is nothing short of strong AI. We’ll get there, but not with generative models.
They did this for Backgammon and found styles of play humanity hasn’t discovered in millennia. Now humans can use those when playing each other and it makes an old boring game feel fresh and exciting.
https://news.ycombinator.com/item?id=32661515
In video games upon release everyone is bad and its an even playing field. But as people practice and gain experience they improve their usage, refine their approaches. You eventually see metas develop and best practices.
This will all be exactly the same. Just give it some time and watch professionals create professional outputs compared to paying for some base level work on Fiverr or whatever.
How's the story there?
I think Copilot is going to live off hype for a while then tank and be looked back on as a failed experiment. Whereas I think that this kind of AI will eventually get to a point where it's extremely useful and could change up certain industries (game assets, marketing materials etc).
I mostly do maintenance of legacy codebases (also known as codebases, lol) where a lot of the work is figuring out where the changes need to be made and actually making the changes is frequently just a few lines here and there.
When I do have to figure out how to use some API, it's often not an open source one, so Copilot would not have it in its corpus.
I think these kinds of conditions are really common since software tends to last for maintenance longer than it is in initial greenfield development.
So I'm confused what kind of work benefits from Copilot. Just pumping out greenfield development of new websites/webapps that don't use much legacy or closed source code or services, just using existing popular open source libraries in commonplace ways?
The other thing I wonder about is code quality. When I look up API docs and stackoverflow examples, I get to read them all, maybe test some examples out in a CLI/REPL, and then decide carefully exactly what to do myself, what special cases to worry about or not, what errors to handle, etc.
Maybe what I end up writing is even the same as Copilot would have written. But in the process, I learn about finer details of the library and make detailed decisions about how to deal with rare edge cases. Might even end up writing a comment calling out a common pitfall I realized might exist.
My question is -- in order to save so much time with Copilot, are you still able to do all this extra thinking and deciding and learning (in cases that warrant it)? Or would doing that just end up consuming most of the time Copilot "saved"?
In other words do you end up producing code much more rapidly, but at the expense of code that looks more like a junior than a senior wrote it, because it is most concerned with working at all, and does not have time to worry about finer details? At the expense of not being as deeply familiar with the foibles of the API you're working on?
Honest questions as I haven't tried Copilot, and these are the thoughts that make me imagine it won't be of value. A lot of what I know I learned from doing the parts of the work that Copilot would be automating. Sure, Copilot would save me time when initially writing it. But would I then have less deep knowledge available when there's a fire in production because I never explored the fine details of my dependencies as much?
One of the major annoyances of working as a team with legacy code is when someone forgets to, or deliberately avoids, conforming their code to the style and techniques of the surrounding code. Nothing grinds my gears like working in a 500 line C++ file, where_every_function_uses_underbars, has consistent 4 space indentions, avoids exceptions, and passes by reference, but right in the middle is that functionThatMiltonWrote that uses camel-case, has 8 space indentions, throws exceptions, and passes-by-pointer.
They have a 60 day free trial. Try it out, it's one of the most interesting changes in developer tools in a while. I feel like I'm living in the future sometimes when using it.
Whoa! Really? Seems like modern software releases would be excited to achieve 98% with the incredible amount of bug fixes/patches released very quickly after the massive beta test known as release day.
Code 0.1% wrong, sends you to the Sun instead of the Moon, debits your account instead of crediting...
Presumably if AI-generated code passes every test case, but would fail on edge cases that some human programmer(s) did not anticipate in their suite of tests, the humans potentially might have made similar coding mistakes as the AI if they had had to personally write the code.
Code that is 98% correct is actually much worse than no code at all. That’s the kind of code that will introduce subtle, systemic faults, and years later result in catastrophic failure when the company realizes they’ve been calculating millions of payments without sales tax, or a clever hacker discovers they can get escalated permissions by passing a well-crafted request etc.
That wouldn't be a meme if it was something that doesn't happen
Currently, working with creatives (including programmers) is a very iterative process for non-creatives.
"I want this."
"No, I meant this."
"Can we try making that line longer?"
"Eh, I'm not feeling it. Why don't we try brighter colors?"
"Ugh. That looks obnoxious. Can we tone down the red?"
etc.
If the non-creative can use this system to reduce that iteration, it will make life easier for them (maybe not for the creative, if they are banking on the hours doing the iterations).
Well, the bottle is open, and the genie is flying around, so we’ll have to see what happens…
https://labs.openai.com/e/bzJBnS0dkgvNeZ681ZTA49nK
Sadly, "highly rendered ai vomit" is filtered.
Camera settings is just a short hand to describe the field of view and depth of focus (at the very least). If you make that implicit you'd still need to give the network the steradians, focal length, circle of confusion, etc. etc. etc. that you want your image to use.
You'd need to understand everything in Hecht's Optics to tweak all the parameters of an AI generated image.
There's nothing special about art. People will still do it as a hobby, like they do with a lot of other dead trades and professions. In many ways I think the personal aspect of the creation and sharing with people you know is the special thing, rather than the creation or performance of a commercial success (not that I've had any experience with the latter, mind you). And I'm not against the mass consumption of art, just that it needn't be produced by people if it has the same entertainment value then that's great.
https://www.davepagecobbler.com/
Repairing expensive shoes is not an automated process. It’s more like fixing a roof leak, landscaping, or changing a flat tire. Jobs for those things still exist and aren’t going anywhere.
Someone is still going to pay for a person to perform or create art for them. Some professional driving jobs will continue to exist long after most are automated. When I said "nobody", that's what's known as hyperbole.
https://melbournechimneysweeping.com.au/ is one near me.
Not wanting to pile on the GP but their point is moot.
To be nuanced, maybe they might have said, "cobblers are less in demand now that many people have moved from owning fewer pairs of shoes they make last through repair to owning more pairs of shoes that they tend to get rid of when they are worn out due to changes in construction materials used in production," but if people have to write like that to make points, nobody will ever make a point.
But to know that these jobs still exist, and might be more common than you realize, now that’s interesting!
That wasn't the larger point. And if that was your issue, why didn't you ask about it so I could have explained it to you better?
The point was that art as a profession and a commercial venture is not special or different from any other industry with respect to automation.
Cobblers are in just as much demand in most of the world as they always have been. They only fell out of demand in car-dependent areas, which is a small minority of the world population (but a vast majority of the HN commenting population since most of the USA outside of a few cities is car dependent)
I don't know if it has anything to do with construction but doubt it. If you actually walk everywhere shoes don't last very long these days, especially shoes under $100.
There are a lot of other urban services that exist in almost the entire populated world, but that most Americans think quaint because they are not relevant to a car dependent highway world.
All that said, this really is a nitpick and the original point stands very well. Some of us just don't like it when car-dependent people forget that they are a small minority worldwide and instead treat urban walkable people as the insignificant minority! Or rather, HN being a forum for all things interesting, we find it interesting to make it a teachable moment. What could be more interesting than finding out that something that has always seemed obvious to you is actually backwards?
By construction, I mean the material and design of shoes people tend to wear. I can't say I've ever met someone who takes sneakers or running shoes to a cobbler and these shoes are more common nowadays.
These kinds of shoes are most of the market because most people don't walk much in the USA. If you walk a lot, you might still not change anything and keep buying the disposable sneakers, throwing away $160 a year.
But if you walk a lot AND are disposed to think critically about the situation, you find that if you pay a bit more for shoes you can make them last many years as long as you resole them periodically. And as I recall, a good $50 sole on a good $150 shoe costs half as much and lasts three times as long as a disposable $80 sneaker.
Not only do you save money (not really a ton) but it actually is more convenient, since even counting resolings, you get more miles between having to go repair or replace your shoe. And you don't have to wear in your leather uppers again. It is truly a luxurious feeling when you come back from the cobbler and have shoes that are worn in and fit your foot just perfectly like a glove... yet the soles are brand new and strong and comfortable and ready for another thousand miles.
Why do you think there's a stereotype of leather boots being popular in NYC? I'm sure the resoleability and longevity in the face of large amounts of daily walking have a lot to do with it.
I think that's why there's a lot of people online who think cobblers aren't a thing anymore. They're from car dependent areas. If you drive everywhere, shoe soles don't wear out much faster than shoe uppers anyway, so it doesn't make sense to care. But if you move to a walkable city, you'll suddenly find it quite economical, since the soles wear out far faster, and the cost of a sole replacement is less than a new pair of shoes, so you might replace the sole a couple times before discarding the shoe.
SD is limited to only digital art first of all, so let's not get too over-excited and hyped around this tool just yet.
You might make painting that looks identical from far away but when you come close it will be completely different.
"Just" need a decent simulation of how that works, first!
People who need illustration or graphics with no particular style can meet their needs with this tool but that is far from art. This replaces commercial illustration, not artists.
It might replace the "nice pic, do you have it at {size} so I can use it as a desktop image" comments on various social media sites.
I don't see it replacing an actual photograph as a wall hanging (or a painting) because they are subtly wrong in some ways. The reflection in the lake doesn't match the landscape... that one cloud has its lighting at slightly different angle than the rest.
Well, they are now. Give it a year and we'll see.
It's not that you can't paint that picture... but you'll never be able to capture that scene in camera. If someone was presenting that as a photograph, it would feel wrong to me because of an understanding of the meteorological criteria for the scene.
I believe that they will generate images that are impressive. I watch https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg and have been impressed with the pace of technology.
Yet, I am doubtful that I'll have a generated image at 11x17 that holds up to the same scrutiny that I apply to my own photographs.
I am absolutely certain that it will be able to generate images that are completely appropriate for images that you don't look at for more than a minute at a time or are used as complimentary material for other content.
All that said, I am not concerned that I will get more or less sales of my photographs with AI generated art competing. The people who are going to pay for a photograph are going to pay for a photograph. Those who aren't - weren't going to in the first place.
40 years ago was 1980. There's a vanishingly small number of fields that have had continuous job security from 1980-2020 (even ignoring Covid).
You might as well say that careers are over for everyone, and have been for a while.
A tiny percentage of lawyers do really well. The rest struggle under crushing debt and insane working conditions.
So you've got one example in the entire economy.
It used to be that you could find work as a political cartoonist and draw a picture of some satirizable politician each day - maybe Dr. Oz with a funny-looking vegetable platter, or Biden falling off a bike labeled "build back better," or something - and that would be a career. Now it seems you can just type "Dr. Oz holding a vegetable platter, political cartoon" into one of these AIs and the bulk of the work is automated. Sure, you could spend the rest of the day refining it, but nobody's really looking for perfection or transcendent skill in their political cartoons.
You can still find work making political videos of Dr. Oz and vegetables (e.g. https://twitter.com/JohnFetterman/status/1564432981841907713). Today's image generation AIs cannot do videos like that. Tomorrow's will, before we know it, and again nobody's looking for more than a baseline level of quality there either.
And even the local newspaper that might have been employing a political cartoonist is being swallowed by high-capital-ownership companies that can replace a lot of the writing with AIs. The expectations of quality are a bit higher - though the AIs have mostly gotten the hang of sports writing - but again people don't expect The Smalltown Gazette to have the standards of The New Yorker.
Sure, the highest-quality drawings, feature films, and longform journalism will get a lot better. And that's great! But most people don't work on such things. What are they going to do? If nothing else, how will they remain an audience with money to spend on the highest-quality works?
(I am not advocating for stopping this process, to be clear. Smashing the AIs isn't a coherent proposal, and smashing the physical machines of automation didn't actually work when the Luddites tried. I'm advocating for admitting that this process is happening and drying up employment, admitting that keeping as much of humanity as possible under a good standard of living is important for humanity, and figuring out what to do about it.)
I've been hearing this for awhile, yet ironically right now we have the biggest shortage of blue-collar workers we've had in a long time.
Good; I hope the future of software engineering AI generative tooling is also more useful for software engineers.
There are already websites which sells tailored "professional" prompts for dall-e, GPT3, etc
Creativity demonstrated by Alpha zero chess engine blows Magnus Carlsen’s mind (from his recent interview with Lex Fridman), I wonder if at some point in the future, we’ll finally throw in the towel and get out of the denial phase.
The fact is, we know no other intelligence except ours. We are the only known being that appreciate art or books, from our subjective conscience. Can AI create its own art, understood and appreciated by itself ? Has intelligence a meaning without « humans » ? It’s part of the same thing. So the AI is modeled after ours. So it’s not its own thing and cannot understand what it creates and if what is created is relevant. AI is just an amazingly powerful tool.
Ditto for me regarding being surrounded by creative people. This isn't going to lose them any business any time soon, not even the younger ones five years from now.
For threatening artists it should be much better. Maybe it's not the model, maybe it's the interface human-computer that fails, but in the end it's what we have.
Maybe someone with a lot of time in their hands can iterate enough times to come up with something nice and depicting what they wanted. Not for me.
It was an absolutely amazing accomplishment. I legitimately thought I could hold out long enough that my next car would be fully self driving. Truckers were an endangered species.
But here we are 20 years later, and we’re still almost there.
We’ve made amazing progress and I love the self driving features I do have on my car, but how many jobs have been replaced by self driving cars?
SQL was designed so that business people could describe queries in English. How did that work out?
Now, of course, we have ORM's, which will get you data, sure, but often in egregiously inefficient ways if not used correctly. If you want to to get it right, you still need to pop the hood and adjust things, and you have to know what you are doing.
I see this as similar.
Sure. Only now the gap has been shrunk by a factor of infinity, because the time to representation was infinite for someone without that experience before.
For example let's say you're a well known designer with a distinct style, and you train your own instance on your lifetime's body of work. Now you can generate whatever you want in 'your style' (just like you can now ask for a painting in Dali's style).
Now you've turned your style and design process into a factory - for every new client you can create whatever they want, in your style, with multiple examples, in a button press. Perhaps you can even sell that 'training library' to other designers?
isn't that Textual Inversion (https://textual-inversion.github.io/ ) ?
It's more or less implemented in some forks (e.g. https://github.com/lstein/stable-diffusion#personalizing-tex... or https://github.com/hlky/sd-enable-textual-inversion (discussed previously ( https://news.ycombinator.com/item?id=32643564 ) )
that's assuming the people artists sell their expertise to are able to understand the difference or what they are looking at
https://www.vice.com/en/article/bvmvqm/an-ai-generated-artwo...
https://imgur.com/a/LexTjLK
This is definitely really important, but these systems generate output that is frequently off or incoherent in really important ways.
I think this is going to lead to some fascinating art, but stories of the death of art are wildly premature.
Liam Kofi Bright outlines this with the Represser archetype in his new essay “White Psychodrama”
I don't think so, because the person I was responding to was flagged. Abusing the flagging functionality to silence difficult opinions about race is not uncommon on this website
Perhaps you should reevaluate your own biases as well.
https://ommer-lab.com/research/latent-diffusion-models/
I'd honestly be interested in your response.
https://waxy.org/2022/08/exploring-12-million-of-the-images-...
> we grabbed the data for over 12 million images used to train Stable Diffusion, and used his Datasette project to make a data browser for you to explore and search it yourself.
> Read on to learn about how this dataset was collected, the websites it most frequently pulled images from, and the artists, famous faces, and fictional characters most frequently found in the data.
LAION collected the images used to train Stable Diffusion:
> All of LAION’s image datasets are built off of Common Crawl, a nonprofit that scrapes billions of webpages monthly and releases them as massive datasets. LAION collected all HTML image tags that had alt-text attributes, classified the resulting 5 billion image-pairs based on their language, and then filtered the results into separate datasets using their resolution, a predicted likelihood of having a watermark, and their predicted “aesthetic” score (i.e. subjective visual quality).
This tech is a huge deal. Huge. Ubiquitous unique, beautiful art. Entire movies made by machines. Too many to even watch them all. Actor models that aren't real people owned by the thousands by people and corporation s.
But maybe the feedback will make marginal defects/artefacts worse. (Which could be manually corrected before re-entering this loop).
Possibly but not necessarily. The human curation prior to posting as well as additional textual context associated with the image could be valuable training signal, even if there is some feedback.
(Also, very happy for somebody to steal this idea!)
I encourage you to check out the subreddit where people share some prompts that you can reuse.
There's usually only a part that describes what is in the image, while half the prompt will be used to describe how does the image look.
Thinking of this one for example: https://www.reddit.com/r/StableDiffusion/comments/wp26lp/i_s... I have reused it successfully.
It's not a search engine. It takes a while to get a feel for what prompts work and how to phrase things. I suggest browsing https://lexica.art/ and using some prompts from there as a template.
Linus didn't invent the core concepts of Unix. He copied the (arguably) good parts from an (arguably) closed ecosystem. His big innovation was leveraging the internet to create a new kind of community not really seen before it. The Linux bazaar gave smart developers excluded from the Bell Labs / BSD cathedral a place to be productive in that style on their own terms and others used it to disrupt the OS business. A lot of shit gets into Linux that the community turns into useful things.
I look at the inputs/outputs of Stable Diffusion and am reminded of the fractal craze of the 80's which revealed the underlying simplicity of incredibly complex looking things. Lots of interesting art and technology came out of that but I don't think anything like the Linux community did.
Stable Diffusion is open source, and was being compared unfavorably to a technically superior but closed source system.
I have seen some very impressive pictures but nothing that seems to "emote"
The rest is up to the skill of the pilot. You can do a lot with color palettes for instance.
MidJourney's production model is good about dirtying things up, you could use it as a post processor. I think its default style of turning everything into fine art-cyberpunk-movie matte painting gets old though.
I've gotten some good atmospheric photorealism results out of DALLE, but currently trying to get outpainting to work to extend them and it's tricky.
Human being never characterize the world with linear rules.
This captures my thoughts very well. That's why these images get old very quickly - you can basically imagine the same thing in your mind. There's no real whimsy, surprise or creativity there. For now at least.
I could see it becoming a marker of low-brow taste where vaguely related illustrations become something akin to decorating your home with velvet paintings of Elvis.
How often have you wanted to hire a conceptual artist? I have never done so in my life. While I have visited many art museums and appreciated it, this has not affected my commercial endeavors.
I think if I was a creative writer or professional artist, the current generation of AI tools could be useful for inspiration. But the bar for quality is just so, so high in the creative industries, I am not concerned that AI will replace them.
It's September 2022 - where is my self driving car?
Nearly impossible because logic doesn’t work in the real world like it does to make pictures.
https://en.wikipedia.org/wiki/Moravec%27s_paradox
https://metarationality.com/rationalism
I can think of two good businesses you could build with SD right now. Neither of them are text to image models or necessarily "art", both involve using it to visualize other kinds of sensing by transforming them to image embeddings.
I don't think AI will replace professional artists, but instead supplement them. Creation of Photoshop, Wacom tablets, 3d modelling and animation tools didn't put professional artists out of their jobs, it gave them new tools to create even better stuff in less time, and I think this is what's happening here too.
All that these algorithms are going to do is slightly bump up the productivity of some people. But that's amazing. Productivity is basically the free money tool. If people are more productive, they get richer for doing less, and the single best thing we can do to improve people's lives is make them richer. Poor people? Make them richer and they'll be happier and healthier. Rich people? Make them richer and tax the hell out of them, make the world better for everyone. Productivity is the best tool we have for making some people richer without making others poorer.
Agree that Stable Diffusion seems a bit over hyped. It is very cool! I am definitely going to play with it. I’m not sure really how much it’s going to change the world - there will be some neat apps and it may put a certain class of concept artist out of business (note I am not saying it will eliminate them - there will always always always be human artists creating wonderful art. But as a business model / way of earning a living, that might change).
I am going to disagree with you on GitHub Copilot. It has radically changed the way I do software engineering on my personal projects and it has increased the efficiency of the engineers at my company by at least 10%, maybe more. You should check it out. It eliminates an entire class of puzzle solving, and the least efficient kind (“how do I drop a column by condition in a pandas dataframe again?”). Simple answer via Google, but maybe 3-5 minutes of reading. 10 seconds via Copilot.
Maybe that's because we never really thought about it. In hindsight, it's only logical. For an artistic rendering, correctness doesn't matter much nor does understanding the model. For flying a plane or driving a car or just transforming code from one language to another, it very much does.
(The “cow tools” Far Side comic is a dumb example.)
Since these AIs can’t count, that extra weird thing is probably going to be people with too many fingers. I actually have a personal collection of weird Midjourney images because if you ask for wide aspect ratios it starts generating multiples of the same thing and fusing them together…
Wouldn't this create more jobs for artists? It's just a different tool
Getting higher level languages and various libraries/frameworks didn't make coders obsolete, because there was still tons of work to do at a "higher level" that required algorithmic/engineering-style thinking. Yes, previously difficult tasks got a lot easier, but that just meant that even more ambitious projects could be tackled, and that was useful.
In contrast, it may be the case with art that the things a relative newbie can spit out with the help of image generation models are "good enough" for most use cases. Yes, a real artist would be able to squeeze out even more, but many companies may be willing to go with the option that's enormously cheaper and still seems to produce okay results.
What we're going to find out is whether novice users of image generator AIs frequently get stuck with inadequate results. IMHO the answer is likely to be "no", especially after we've iterated on the tools for a few more years.
It's not clear that you'll see something similar here, that easier-to-produce high complexity art will greatly increase demand for said art.
It's like some of the stuff that GPT-3 generates, it sounds extrememly plausible and realistic but then you read it properly and realise that parts just don't make any sense.
However I don't think these things are show-stoppers, I think we'll see a lot of artists getting good at providing the "feedstock" and guidance to these systems to deal with those quirks and generating some really interesing work.
Symbolic AI dominated the field for 50 years: thanks partially to an accident of history (the highly influential work of Minsky, Newell, etc) and partially due to lack of the data needed to try anything else.
Now we’ve seen essentially 50 years of research on data-driven methods compressed into 5. In retrospect it makes sense that applications to which symbolic reasoning is especially ill-suited (including artistic rendering) would be the lowest-hanging fruit.
Francis Bacon has a quote that is apt about how he paints the things that cannot be described with words.
Eg with other AI problems out there I can see a potential application to medicine, self driving cars etc, but I just don’t see what the bigger goal of this is going to be.
"elon musk giving donald trump a massage using pizza sauce instead of oil, in a majestic room filled with flowers and golden toilets"
...and the result was a crappy AI-generated picture of not-quite donald trump holding a terrible rendering of a pizza, and some hands sticking out of random places. There were some red flowers in the picture at least.
I realize this isn't a "serious" use case, but clearly the tech isn't doing what I'm hoping it does.
I tried "coffee beans with cartoon mouths" and it's just a picture of some coffee beans. I don't get it.
"A cup on a plate".
And then replace cup for other objects. It generates nonsense pretty quickly. It seems most able to generate stuff close to something which already exists in a complete form. Sort of a pastiche machine.
That's the absense of reason sticking out: the "ai" mimics, but doesnt understand, so its creations are frankenstein-like bodies, almost human and for this reason more revulsing. So this "ai" represents an insane painter.
I wish people would stop exaggerating or making statements without some insight behind them.
I've seen hundreds, if not thousands of photorealistic result that range from acceptable to remarkably good.
One example it took seconds to find:
https://lexica.art/prompt/3fbc30ee-ca0f-42f6-8ea5-6890d469ab...
> [...] It takes a while to get a feel for what prompts work and how to phrase things. I suggest browsing https://lexica.art/ and using some prompts from there as a template.
I think this applies to all AI generated images though.
https://cdn.discordapp.com/attachments/999426920376717513/10...
So Donald Buck who is a duck is too derivative, but Ronald Cluck who is a bear might be fine.
Depending on how close it is it may not even be legal. To give a music analogy: at least in my country, if you take a melody, change all the notes' durations, and transpose it you'd still infringe
I can write a book about a boy wizard's adventures wizard school and that's legal, but if I call them Harry Potter it isn't.
I can create Harry Potter fanart and distribute it online pretty freely - but slap it on a mug and sell it, and that's illegal.
I can record an audio description of a painting that's as detailed as I like and it's legal to distribute - but take a photograph of the same painting and it's a derivative work, no matter how artistic my choice of camera settings.
We don't really have any prior examples that are precisely like these huge ML models trained on copyrighted data - and depending on which imprecise analogy you choose, you can come to a different conclusion.
If your fan art is infringing, it was infringing whether or not it was on a mug or on dropbox.
Photograph one is not true. For commentary it can be by audio, or printed on a mug or whatever, commentary is transformative. Go out and take a photo of the world outside, if you live in a city you've captured thousands of copyrighted materials in your image. Maybe it captures someones painting, maybe it doesn't. Whether its fair or not is if it's transformative, the format doesn't matter.
I'm not seeing an ethical difference from any of this. Or did I miss the point?
Perhaps I should have said Darth Vader, then - the point is you can copyright a character independent of the copyright on a book's text, and the trademark on the series name, and the fact that broad concepts like "black-clad masked evil overlord" are uncopyrightable. And that copyright can persist even if you transform a book character into an engraved coffee mug.
> I'm not seeing an ethical difference from any of this. Or did I miss the point?
The difference is:
If a person says "Stable diffusion is to its copyrighted training data as an audio description is to a painting" or "Stable diffusion is to its copyrighted training data as the concept of boy wizards is to harry potter" they would probably say it's ethically fine.
If a person says "Stable diffusion is to its copyrighted training data as a photograph of a painting is to the painting" or "Stable diffusion is to its copyrighted training data as video lecture is to a single image in its slides" they might well say it's not ethical.
So since automated driving will result in financial suffering to those working in trucking and Uber and whatnot, automating driving is not ethical?
Think of this instead: a doctor collects a big volume of symptoms and analyses and creates a statistical way to cure people more easily. They publish many examples of their work without licensing anyone (legally and morally) to use it freely. Now some algorithm collects their data and many others data and transforms it into a better method. A doctor suffers from going out of business. Is that ethical? On one hand, the algorithm invented something new and easier to access. On the other, it basically stole parts of their and similar researches on a previously unthinkable scale. We humans copy ideas all the time and this is somewhat normal, but this enormous at-scale capability was never a thing.
Personally I don’t care for optometrists, uber drivers or designers. Nature will find a way. But when we talk about fundamental social contracts like property or accumulated knowledge protection, I think it is unethical to break them, regardless of technicalities. If it’s such a great advancement benefiting everyone, why can’t AI creators just ask permission for 2.3B of datapoints they used?
I’m not optimistic that their impact will be positive though.
BUT it will likely be disrupting the upper middle class more than the lower middle classes, as it's mostly disrupting to creative work. Upper middle class people have a louder voice.
With the risk of looking silly, I declare "meh" once more, just as I "meh"-eh when GPT-3 came out.
The so-called "AI" is not fundamentally different from the AI of the 80s, it's just that now we have much better hardware. The main problem of the past AI winters still remains - the existing algorithms focus on statistical methods, which can be rather inexact. Imagine a nuclear plant controlled by an AI, or an airplane flown by a neural network. They completely lack reasoning capability and therefore you can't trust they will be able to adapt to unpredictable situations.
Either way, I think the current "AI" is fundamentally limited by the statistical approach to problem solving. Without any reasoning capabilities, no amount of incremental improvements will change the fact that neural networks are simply making guesses based on existing data sets. Nothing magical or revolutionary, it's the same thing we've known for many decades.
Not lifetime. Right now. Just one week of StableDiffusion in the wild has seen the development of many clients, GUIs, optimizations, plugins for commercial software, editing workflows, the list is endless.
The speed of it all is dazzling. It will not take long before this is assembled into a smooth experience that runs everywhere, with obvious end state being your phone.
And that's still just artistic image generation. The open sourcing of it means people can make any vertical, precisely optimized for a particular (commercial) domain.
Almost all shortcomings seem to be crushed in no time. Bad at drawing eyes? Here's a new encoder that fixes it.
The creator of StableDiffusion indicated to soon tackle music generation and if I remember correctly, even poetry. And there's 3D and video.
The ultimate end state: if you can imagine it, it can generate it. Not only are we much closer to that point than society realizes, we're also moving at an exponential speed towards it.