"We’ve heard from regulators and the general public that we need to focus more strongly on security to ensure that we’re taking all the steps possible to make sure people don't use Stable Diffusion for illegal purposes or hurting people."
"What we do need to do is listen to society as a whole, listen to regulators, listen to the community."
"So when Stability AI says we have to slow down just a little it's because if we don't deal with very reasonable feedback from society and our own communities then there is a chance open source AI simply won't exist and nobody will be able to release powerful models."
Two days ago:
“Nobody has any voting rights except our employees — no billionaires, big funds, governments or anyone else with control of the company or the communities we support. We’re completely independent,” Mostaque told TechCrunch in a previous interview. “We plan to use our compute to accelerate open source, foundational AI.”
Isn't the whole point of open source, so long as licenses and attributions are respected, that anyone is free to do whatever they please with these models and their redistribution?
That’s the FSF definition yea, but lots of developers nowadays don’t agree with the FSF. For example some people don’t want companies to use their open source libraries/code for a profit.
> some people don’t want companies to use their open source libraries/code for a profit.
IMO the AGPL goes a long way to solving that problem. But if the AGPL is not for you, I suppose you could use some non-commercial license terms. It seems like "closed source" is a much better fit for folks who want a great deal of control over licensees. In practice, "closed source" code can be published for licensees to see but instead of granting terms to all comers you could force people to ask you for a license, review their use case and only then decide to grant a copyright license -- with or without source.
AGPL doesn't work for modern day problems. The problem you have is that you are developing an open source bit of infrastructure and your only method of profiting is your own hosted version giving people the option of getting it for free and managing hosting, or letting you do it for a small profit.
Then Amazon comes in and sets up their own even easier hosting while providing none of the profits or development effort back to the original project. The AGPL is no problem because Amazon is fine to publish any changes they make, thats not an issue because the real product is the hosting and not the code.
That's not Amazon using open source code for profit. They make money from hosting. They don't make money from the code.
The problem is releasing stuff as open source and expecting to profit off it while preventing others from doing the same. You fundamentally can't. Why bother with open source with an attitude like that?
They don’t. That’s why they came up with a new license that still allows self hosting and viewing the source but doesn’t allow cloud providers to capture their income.
"AGPL doesn't work for modern day problems. The problem you have is that you are developing an open source bit of infrastructure and your only method of profiting is your own hosted version giving people the option of getting it for free and managing hosting, or letting you do it for a small profit."
AGPL is for code, not for infrastructure, and you can still profit off of AGPL-licensed code by:
1 - selling support for it
2 - getting donations
3 - asking for money in return for working on feature requests
4 - being paid by a corporations to work on an open source product they themselves use and profit from
> Then Amazon comes in and sets up their own even easier hosting while providing none of the profits or development effort back to the original project
There's nothing particularly "modern" about this "problem". Redhat caused this problem with GPL and BSD-licensed code in 1993.
It's a feature, not a bug. Some people probably didn't like it then either.
If you resent companies and people making money from your work, you're not really embracing the ideals of open source. So just keep it closed source. Like I said earlier you can always reveal the source to licensees when they pay you, and you can prevent them from distributing the source.
1) Who is Daniel Jeffries? There's no explanation of how he's related to Stability.
2) StabilityAI gave RunwayML compute time for them to train Stable Diffusion (they're also the creators of the original model). It's weird to categorize them as " other groups leak the model". They're the ones that created the model! (Source: https://huggingface.co/runwayml/stable-diffusion-v1-5/discus...)
The discourse has already changed quite a bit since the first release, which was only 2 months ago, and is getting alarmingly close from OpenAI's "we must delay release of XXX for safety reasons". It was probably to be expected, OpenAI are not just morons who decided to freeze opensource progresses, there are likely legal reasons behind it. But adding to that last weeks dramas, I am not very bullish on StabilityAI, hope I'll be proven wrong
1. The web UIs I have used are taking advantage of the same mental pathways as an electronic slot machine. Just like you can max out your bet on a slot machine and mash a button until you run out of credits, you can do the same on the hosted stable diffusion apps until you get a shareable hit.
2. Just like the dream you had last night, nobody wants to hear about it at breakfast, no matter how epic it was, because it's not backed by any meaning.
That said, I love stable diffusion and am an addict to it almost every day.
TBH, I bought a 3090 and it broke after a week so I sent it back. It was about $1200, and I'm down about $80 on the web site at this point, so... I have been pondering a cloud solution though, but haven't done the math all the way yet. It's a bit like buying a $15K boat and then just going out on it once a month.
Slower means around a minute per picture. It's amazing! Guys with GPUs more expensive than my whole computer can't run it, but a laptop without fans can. Well done Apple.
I have a old ass GTX1070 and run it just fine, it's not the fastest but it works. A lot of these cases are just PEBCAK's and should just be removed from the internet, it's been made so easy that a literal child could install and use SD.
Well as long as they have a recent-ish NVidia card, rip AMD users.
The 3090 I had was actually somewhat slower than dreamstudio.ai, but the nice thing was the set of python tools I got with it was scriptable so I could do things like create an image, use that as input for another one, and so on, then make them into an animation. There's a bit of tax with initializing python and loading the ML model with each iteration, but if I ever get my hands on a 4090 I'm sure I can solve that.
I disagree with your point 2, to a greater or lesser extent. People have made some amazing images with the technology. From throwaway funny stuff which will probably contribute to 'meme' culture, to legitimately beautiful images that I would be happy to have on the wall.
I do agree that most of the dross I create is only of interest to me. OTOH I got some laughs with my "Liz Truss holding up signs with rude words on them" series from British friends yesterday.
"Just like the dream you had last night, nobody wants to hear about it at breakfast, no matter how epic it was, because it's not backed by any meaning."
Who stays awake at night wondering what meaning that cool picture of a dragon had?
They just enjoy it and move on with their day.
Same with AI-generated images... they just look cool, amazing, or hot.. whatever. Most people just enjoy them, without wondering what deep meaning they might have.
It's mostly art critics and the like who wonder what deep meaning Van Gogh's Starry Night or the Mona Lisa have (never mind abstract work like Jackson Pollock's). Everyone else just likes it or they don't.
His comments regarding RunwayML’s release of 1.5 were especially interesting:
> “No they did not. They supplied a single researcher, no data, not compute and none of the other reseachers. So it’s a nice thing to claim now but it’s basically BS. They also spoke to me on the phone, said they agreed about the bigger picture and then cut off communications and turned around and did the exact opposite which is negotiating in bad faith.”
> “I’m saying they are bad faith actors who agreed to one thing, didn’t get the consent of other researchers who worked hard on the project and then turned around and did something else.”
>We want to crush any chance of CP. If folks use it for that entire generative AI space will go radioactive and yes there are some things that can be done to make it much much harder for folks to abuse and we are working with THORN and others right now to make it a reality.
I agree that if all CSAM was virtual and no IRL abuse occurred anymore, that would be a vast improvement, despite the continued existence of CSAM. But I suspect many abusers aren't just in it for the images. They want to abuse real people. And in that case, even if all images are virtual, they still feed into real abuses in real life.
People don't want to seriously grapple with these sorts of harm reduction arguments. They see sick people getting off on horrific things and want that stopped and the MSM will be more than eager to parade out a string of "why is [company X] allowing this to happen?" articles until the company becomes radioactive to investors.
It's a new form of deplatforming - just as people have made careers out of trying to get speech/expression that they dislike removed from the internet, now we're going to see AI companies cripple their own models to ensure that they can't be used to produce speech/expression that are disfavored, out of fear of the reputational consequences of not doing so.
This is advocating for increasing the number of victims of CSAM to include source material taken from every public photo of a child ever made. This does not reduce the number of victims, it amounts to deepfaking done to children on a global scale, in the desperate hope of justifying nuance and ambiguity in an area where none can exist. That's not harm reduction, it is explicitly harm normalization and legitimization. There is no such thing (and never will be such a thing) as victimless CSAM.
This is hoping for some technical means to erase the transgressive nature of the concept itself. It simply is not possible to reduce harm to children by legitimizing provocative imagery of children.
It is an interesting ethical question that needs more research done in to it. Though given how people's brains tend to shut down upon hearing the topic, I feel like the general public would be vastly opposed to it even if it was proven to lead to better societal outcomes (decreased child abuse).
there is a history to that. In some US states it became illegal to have cgi images of CP , Secondlife had that problem and they still ban it. I think it got turned down on free speech grounds but there are still some kind of restrictions
Those restrictions made sense in a world without Stable Diffusion because CGI images were thought to stimulate interest in photorealistic CSAM and photorealistic CSAM couldn't be acquired without outright acquiring actual CSAM.
Now that we can readily generate photorealistic CSAM, there's little to no risk of inadvertently creating an customer base for actual CSAM.
The only way to do that is to not release the model, not let people run the model locally, and basically turn it into DALL-E with a web UI and rented compute. Which defeats the entire point of Stable Diffusion.
You can use AI in two steps, one for the general image and the second one to change the age, with access only to the crop of the head. There was a novel by Asimov where someone did a crime by composing the actions of two robots, who were doing perfectly legal things each one separately.
>> We want to crush any chance of CP. If folks use it for that entire generative AI space will go radioactive and yes there are some things that can be done to make it much much harder for folks to abuse and we are working with THORN and others right now to make it a reality.
I'm absolutely certain Linux has been used to kill children. Detergent, pesticide, and even pillows too!
Tools shouldn't be limited based on what the worst way they can be used. Stable diffusion is an absolute positive for society. Even when it's used to generate CP, every image that model creates is not one that involves a real kid.
The cat is out of the bag when OpenAI announced DALLE. Stable Diffusion only accelerated it a bit. Even if Sablity or lawmakers manage to prevent or outlaw open models, criminals will continue to build their own and ignore laws.
The only thing their reluctance does is harm Stability and give their competition a chance to catch up. Perhaps that's a good thing. Maybe it's time for another organization to take the lead.
> criminals will continue to build their own and ignore laws
More specifically, researchers and hobbyists would be made criminals in this hypothetical. Why would I stop playing with ML just because someone passed a useless and poorly thought out law?
You can make child pornography with a can of paint and your hand. What a totally laughable farce and excuse. People are making (notice the tense) REAMS of pornography with AI just as they have with literally every technology that has ever been invented to produce creative works.
I just had the (very unreflected) thought that generated CP is better than sharing actual victim's pictures? Might be too short-sighted, didn't think about it that hard. But (preventing CP by using) this tech will not prevent CP nor sexual assault of minors.
I get the dilemma as a creator though. I wouldn't want my products to be used that way too.
If the results don't have enough artifacts to identidy it as ai generated, that could give cover to collections of images where actual minors were involved. Not great.
I'm very interested in this tech and have been working with it. I have heard tell of a CSAM model. I wont say anymore because I don't want to feed that hideous fire.
I would add though - if Moore's law continues this will be almost unstoppable in a decade or too.
I'm impressed by how this panned out. A few months ago, before things had worsened this much, Emad(Stability CEO) was talking on discord how happy he was about the inpainting model being trained and released soon.
At that point, I assume RunwayML too talked about releasing it.
Based on a Reddit post [1], the author of this is Stability AI's chief information officer.
My very rough take on the situation: the company gained their notoriety by building on OpenAI's pioneering research but with an important twist of releasing their models as unneutered open source. Now, their openness is starting to falter due to strong pressure from outside forces.
If they're unable to continue playing the hardball game they themselves invented, I think their glory days will end as fast as they started. The competitive advantage was always their boldness. If they lose that, quickly others will take their place.
In general, I don't think tech that's as open, powerful and easily reproducible as these language models can be stopped. Sure, maybe regulations will delay it a bit, but give it a few years and any decent hacker or tinkerer will be dabbling with 5x better tech with 5x less effort.
You're missing an important vulnerability of this tech. The model was trained on copyrighted material. There are enormous pressures to close and stop this.
You can't predict future legislation. Intellectual property legislation (which is an absolute cancer IMO) can outlaw models and their results. It can outlaw distribution of the data sets, the training, the models. Tech companies already acted way beyond the requirements of the law and effectively censured open source projects like popcorn time.
Can they prevent a determined hacker which already got everything? No.
Could this be the last model to be trained on a wide dataset, available to the public? Yes.
Could they make it a living hell where getting these tools in the future will only be from untrustworthy websites where half the download buttons give you an exe, and all your less tech savvy friends won't bother? Easily.
Could this tool become impossible for companies to use without risking litigation? Very easily.
People tend to forget those making the rules do not have their interests at heart, and every single intellectual property law is designed to leave companies and not people holding all the rights. And those laws can absolutely do damage. Do not underestimate the power of legislation.
You're thinking copyright liability[0], but the real worry, straight from the mouths of the Stability people[1], is AI-generated CSAM. That will make the whole field of generative art legally radioactive.
At least with copyright law, there's an argument for training being fair use. If generative art becomes a notorious market for CSAM, everyone in the field goes to jail.
[0] Also, I'd like to know what your opinion is on GitHub Copilot. A lot of people decry Copilot for stealing code but love Stable Diffusion for being public, even though they're the same concept and trained in the same quasi-ethical way.
This area is already well explored just with fake CSAM generated by artists using photoshop, cartoons, etc. The modern thinking is that it supports and encourages a behavior that can lead to actual violence.
If you constantly watch videos of people eating cheeseburgers, you might want to eat a cheeseburger yourself.
> The modern thinking is that it supports and encourages a behavior that can lead to actual violence.
Hasn't this nonsense been thoroughly debunked by multiple studies at this point? I would assume evidence and "modern thinking" supports the exact opposite of what you claim, unless by modern thinking you mean the same thinking that tries to hide research they don't like.
The pleasure centers activated by videogames and pornography are quite radically different; I would not assume that the reactions to simulated sexuality is the same as simulated violence.
Then ban porn especially skits that depict actions that society deems deplorable like suffocation and rape, or are the pleasure centers for those also different.
> I would not assume that the reactions to simulated sexuality is the same as simulated violence.
A would not assume anything. Conduct research and draw conclusions. Don't speculate.
People who had no clue what videogames were, were the ones arguing that playing a violent video game would make you want to commit actual violence. The counterargument that players made was that they could "tell reality from fiction" - i.e. that when they played Mortal Kombat or Call of Duty, they put their "Real Life" brain away and put on their "Fictional Video Game" brain, so videogames can't make people violent.
This is the right conclusion, but the logic is entirely wrong.
The reason why video games do not cause violence is that play violence is not anywhere close to the real thing, not that people firewall off fiction from reality. There's plenty of cases in which a piece of fiction has changed people's views! Crime shows are notorious for skewing how actual juries rule on cases. Perry Mason[0] taught them to expect dramatic confessions and CSI[1] taught them to weigh whiz-bang forensics over other kinds of evidence.
In the specific case of porn, there isn't really a difference between "play sex" and "real sex": they poke the same regions of your brain. And the people who are responsible for keeping actual pedophiles from reoffending are pretty much unanimous that the worst thing you can do is give them a bunch of, uh... let's call it "material". So if you're already a pedophile, giving you access to simulated CSAM won't substitute for the real thing. It'll just desensitize you to reoffending.
A lot of claims and no supporting research. My position is clear: You need to give clear evidence that X causes harmful Y before we can discuss banning X. We don't ban X because you and I find it deplorable.
>> Conduct research and draw conclusions. Don't speculate.
>If you constantly watch videos of people eating cheeseburgers, you might want to eat a cheeseburger yourself.
this is retarded, if you watch a movie, play a video game, read a book with crime events then you will become a criminal. We have a ton of shooter games and still no evidence that this caused more gun violence around the world.
Just like how the incredible availability of porn on the internet has led to millennials being the generation that has the most sex ever: https://news.ycombinator.com/item?id=12433236
There are way too many factors at play to simply point at porn, which is probably harder to obtain now in all honesty. I found many random porn magazines/pages as a child. Never did I ever go looking for it, but finding it was always a thrill.
People buy less magazines now (based on convenience store shelves increasingly excluding them.)
>This area is already well explored just with fake CSAM generated by artists using photoshop, cartoons, etc.
I'm familiar with the research in this area, and that's not something you can say confidently; most work (and by that I mean 2 or 3 papers in total) has gone into investigating the role 'generated' depictions of CSAM play in the collections of hoarders. No psychological study, as far as I'm aware, has conducted an investigation on those who enjoy cartoon material akin to what you might find in a Japanese manga.
In fact, there's some evidence against what you're saying; anthropological research on fans of cartoon material ('lolicons' or 'shotacons') in Japan shows that their communities draw hard lines between '2D' and '3D' not just in this area of sexuality, but in their sexualities as a whole. This sexual inclination toward the 2D world is termed the 2D-complex and is akin to 'digital sexuality' or fictophilia, not pedophilia.
By way of analogy, perhaps BDSM would work as a good counter point to you. Many people (some studies suggest the majority of people) engage in 'rape fantasies' or other such fantasies of illegal or immoral nature, yet although actual depiction of rape is rightly banned by the state, its simulated variants are not, and we are comfortable to acknowledge that sexual desires do not always manifest in real life, and sometimes the thrill of fantasy itself is the attraction. To make it real would, ironically, defeat the whole point.
One issue with this is that the fake CSAM wouldn't be the cartoons of a Japanese Manga, it would (or could) be photo realistic. It could be photo realistic of real children. This is obviously possibly bad because it might fuel or encourage pedophiles, but it also has lots of other negative possibilities too.
One example of a bad thing - you could easily imagine an instagram bot that looks for pictures of people with their kids, then uses a Stable Diffusion like model to produce pictures of the people having sex with their kids, or horrible things happening to the kids, and reply to the target account. The bot might threaten to post the pictures and accuse the person of being a pedophile unless the person pays X in bitcoin (or whatever). Or, the bot could just post such pictures for fun.
I think we don't know if fake CSAM will have a good or bad effect on pedophiles and, sadly, there is no real way to reliably test that (so far as I know). Fake CSAM might placate pedophiles, or it might whet their appetite. It's hard to know what to do.
I think we will eventually get to the point where very good unrestricted image generation models are available to the general public. When that happens there will be chaos - you will live to see man-made horrors beyond your comprehension.
That's a good point, and I agree - I only wanted to pick up on the point about 'cartoons'. As for whether realistic generated images with real human data sources would have a good effect, it certainly wouldn't have a good effect on preventing further child abuse, and again, as far as I know there's no evidence that it would 'placate' them in the sense of the (widely debunked) catharsis theory.
And of course, I'm not looking forward to the world ushered in by free roam with this technology, mainly for the reasons you stated.
Modern thinking doesn't mean evidence-based thinking. To the contrary, it gets even more politicized, rather than becoming more evidence-based. Here's an evidence-based counterargument [0].
If a boy constantly watch media of pretty girl, he may want to become a pretty girl himself. Which is fine IMO but traditional parents are not worried about this possibility much.
So long as the AI can remain creative. Once it has exhausted itself of that while the niche consumers still crave for more variation. That's when children start to get hurt again.
They were shamed into working with a nonprofit aimed at protecting children (Thorn) whose executive director stated publicly at the Stanford conference a few weeks ago that her organization is against the concept of synthetic images.
We can likely make the very strong assumption that the training data didn't contain any CSAM so it would be more difficult for the model to produce CSAM. Also, I would imagine they trained the model without porn too, so inferring CSAM based on legal adult porn would also be quite difficult. Am I missing something?
Not only that, but with techniques like in-painting you could start with something that wasn't CP and then progressively make the model generate parts of it which then make up such an image right now. Stability saying they want to release an open version of SD that can't make CP is like a pen maker selling a pen that can't make CP: horseshit.
Actually I saw Stable Diffusion-generated semi-stylized / semi-photorealistic (kind of like photorealistic-ish anime) CSAM on 4chan literally a day or so ago when I randomly decided to go to 4chan and saw that AI art threads are super popular there right now.
Keep in mind, there already are a lot of illustrated/anime style pictures of CSAM on that site of years though (something that is legal in many countries), so it's sort of becoming a blurred area as these AI art generators are still somewhat like that but now are getting to be more photorealistic.
As far as the models not being trained on NSFW content, there was already leaked models that were, and there are unofficial models trained by outsiders using SD that are specifically trained on for example adult image websites.
What an utterly predictable development. I was happy that Stability put their model out there without any waffling about "concerns" and "communities", but I was always skeptical they'd last. And well, now they're folding like cardboard when faced with a criticism that they should've seen coming. The most concerning thing here is that there's no conceivable approach they can take to prevent CP while keeping their model open; either it is open, and people can use/re-train it to make CP, or it is closed.
The way I see things, it all starts from the interests of the participants. Stable diffusion got their publicity from opening their model, but their interests are squeezing a maximal profit from it. And then there is Dall-E and midjourney, with similar incentives.
Then there are narratives. They are weaved so that the suggested actions and solutions will somehow fit the interests of the participants. The narrative can be CSAM, it can be copyright of artists and owners of the training set, the narrative can be disinformation. The narrative doesn't care that current laws do not prohibit anything and that it's all legal. The narrative justifies actions the participants wanted to do because of their interests.
And finally there are actions. They can push legislations, but that's not the only tool (and yes it's slow). Companies can always comply and cooperate, especially when their interests align. Google itself is a participant, with Imagen. They can create a restrictive policy and kick things off their search engine, because that is in their interests too, not because of a narrative or legislation. Just like they profited in YouTube for every piracy site suppressed.
The interests of every single company is stacked against individuals running this at home for free. There are enough narratives to be weaved to justify actions which would stop that.
For decades, and in many countries even today, just getting paid to drive someone in your car is illegal, and you need a "taxi license". It doesn't need to make sense. We could end up with required license to use generative AI in 10 years and nobody would bat an eye after decades of propaganda and narratives.
I thought there was a case that "virtual" images were already legal. Does that not apply here because real images are used as the training dataset or something? If no illegal images are used as input, I don't see how the output could be (or should be) illegal. There's no nexus to any real person being harmed.
You'd need to decode those latents back to an image representation and scan. (possibly other ways but that's the most straightforward I can come up with although time intensive).
All this is extremely interesting to see how it will unfold. You know what's also trained on copyrighted material? Real (human) artists! Not only trained but also constantly draw inspiration directly from copyrighted material.
This is my favorite part of this whole "debate". I think the layperson does not realize how "risky" it is to be an "artist" because of copyright.
The entire art tradition is based on copying for study, and then using our brains to convince ourselves that we've "transformed it enough" or "my reference is obscure enough that no one will know."
Now we've simplified that process, and more people are exposed to the risk. I hope that the law takes a minute to evaluate the pace of change instead of saying "ITS TOO DANGEROUS, we must BAN IT", but my hopes are low.
I couldn't have said it better myself. Pretty much all of art relies on inspiration and "inspiration". Pending few possible cases (I've yet to see) where actual art is lifted and pasted, all I've seen so far (firsthand even) is style lifting and even that not fully. Is style copyrightable?
Some very famous paintings it can almost reproduce, like the Mona Lisa and The Last Supper. It can’t get it quite right but I think it’s close enough to be considered copying. So there might be a copyrighted instance of that, but I haven’t seen one yet.
Surely eventually, hardware will catch up and in ten years people will be able to create models like this on their home PC, and people will just share PBs of images as datatasets/models like with normal piracy?
> The model was trained on copyrighted material. There are enormous pressures to close and stop this.
> getting these tools in the future will only be from untrustworthy websites where half the download buttons give you an exe
These models can already be downloaded via well known (ie community reviewed) torrents. So can many terabytes of labeled training data. This particular horse is well out of the barn.
My network connection is horrible for large http downloads but torrents work fine. Can you provide guidance on where I can find these torrents? It doesn't have to be direct links, just a hint that can help me find them.
This NovelAI guide is a good starting point. https://rentry.org/voldy If you're interested in training data then reading about what was used and where it was sourced from would be informative. As to torrent indexing services, well, I dunno if I'm supposed to link those here but they're easy enough to find and there are a lot of them out there.
I'm not a data hoarder, but from the moment Stable Diffusion was released I had a gut feeling that I should download everything available while it's there.
Somewhat similar gut feeling to when popcorn time was released, although it might not be exactly the same.
While I really wish I'm wrong, my gut tells me that broadly trained machine learning models available to the general public won't last and that intellectual property hawks are going to one day cancel and remove these models and code from all convenient access channels.
That somehow international legislation will converge on the strictest possible interpretation of intellectual property, and those models will become illegal by the mere fact they were trained on copyrighted material.
So reminder to everyone: Download! Get it and use it before they try to close the Stable doors after the horses Diffused. Do not be fooled by the illusion that just because it's open source it will be there forever! Popcorn time lost a similar battle.
Get it now when there are trustworthy sources. Once these kinds of things go underground, it gets much harder to get a trustworthy version.
Can't find a working link for 1.5, but for anyone who cares, I did turn up https://rentry.org/voldy, which has a actually-working link for SD 1.4[0] (in addition to the huggingface paywall) as well as to a couple other compatible models including NovelAI.
> You have to create an account and agree to some stuff and be logged in
So a paywall. I agree it's a annoyingly misleading term for the concept and would be happy to hear better alternatives[0], but I haven't found one yet.
0: like eg "passphrase" instead of "password" or "assuming the conclusion" instead of "begging the question" for their respective concepts
I would disagree with their assertion that requiring an account is a paywall. I mean, it's in the name... if you don't have to pay, it's not a paywall. A barrier to entry, sure, but a far easier one to overcome than entering payment details.
> These models may contain executable code. Beware of malware.
Seconded, actually. I do have a bad habit of assuming people already know this.
> Official sources are recommended for this reason.
Very not seconded; see for example comments elsewhere in this thread about untrustworthy sources for popcorn time, and recall that the GP was specifically discussing the risk of Stability AI deciding to kill this.
Okay, bad habit stikes again apparently. To be explicit: If you get it from official sources, you should still beware of malware just as much as if you got it from a skeezy darkweb site (and vice versa).
You mention Popcorn time. I wonder if torrents in general could be a great example of how something like this plays out? Torrenting took the world by storm and had an amazing "product-market fit" for the early internet days. Of course, downloading copyrighted material was always illegal but that didn't stop many.
Over time, legal but paid alternatives rose up: Spotify, iTunes, Netflix. These players found their place in the market by balancing the interest of copyright holders and the needs of users looking for cheap and easy access to entertainment.
Just as Netflix acquired large content libraries, same here. With enough money, large training datasets could be acquired in a legally solid manner.
It's interesting to think where this analogy might fail as well, and how the paths of these technologies could differ. For one, torrenting was mostly for entertainment, and thus impacted B2C first. On the other hand, language models are more so for media _creation_ and the B2B sphere.
They can and do fight dirty. They don't only use legal tactics, they use legal options to get the information off from trustworthy sources.
Like torrents, you first have to resort to random websites who get randomly taken down as they acquire reputation. If a person takes the face and responsibility for something, he gets litigated into oblivion.
So you get to the point where trustworthy and untrustworthy sources are indistinguishable
.
Now what they do is create untrustworthy sources. Like time for popcorn. Sow discord.
Fork several times, create intentionally malwared versions of both the program and the website. Keep kicking off the trustworthy sources of search engines, while magically skipping takedown requests for the less trustworthy websites.
Find ways to break old versions if possible, just to force them to keep moving. (they can make gradio randomly change APIs just to break the old trustworthy versions)
Fight dirty, you say? Capital idea. Let's see, the principals of any company or trade group that try litigating model providers "into oblivion" can have some more legal fun when synthetic images of them diddling their kids find their way to law enforcement or local vigilante organisations. And for anyone too squeamish for that, there should be a way to use the tech to do a really good SWATing for suspected murder -- bit of fine-tuning on screenshots from ISIS and Mexican drug gang videos, etcetera.
Before this gets flagged to oblivion, this is obvious. You just have to recognise that the "regulators" and industry insiders Emad is trying to "shield" you from are enemies and ask yourself, how do I hurt them?
"Over time, legal but paid alternatives rose up: Spotify, iTunes, Netflix. These players found their place in the market by balancing the interest of copyright holders and the needs of users looking for cheap and easy access to entertainment."
You didn't mention one of the largest (perhaps even the largest) distributor of copyright content (which happens to also be free, for now): YouTube.
You can watch/listen to endless amounts of copyrighted content (and other types of content) on there completely for free, and to say it's tremendously popular would be an understatement.
Google has made it work through ads. Perhaps something like that will happen with image-generating AI.
Notably Youtube contained rampant copyright infringement in the early years, but to be able to hold their market position gradually pivoted to a system that treats existing large copyright holders preferentially and clamps down on everyone else.
Here's a (not-recommended but amusing) nuclear option:
Tit-for-tat. Regulators and artists don't want this? Okay, include in all open source software licenses that regulators and artists are now barred from using them without payment.
Definitions are what matter, they're the reason you can use words and I can understand what you mean by them.
"Open source" has a definition (https://opensource.org/osd) and the GPL meets it, because it doesn't prevent derived worries from being distributed under the same license.
Interestingly it's the same with attribution requirements for art, but since it's not written words, nobody can claim: "this part is exactly my GPL code".
But with art it's "this is exactly how I do texture on metal", "this is exactly how I paint steampunk greeble", "this is how I do clouds" etc.
> [...] is a nonfree license because it extends the four freedoms only to some kinds of organizations, not to all. Such a restriction in a software license, in the name of any cause whatsoever, imposes too much power over users. Please don't use this license, and we urge you to avoid any software that has been released under it.
> The license shall not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources. The license shall not require a royalty or other fee for such sale.
> Rationale: By constraining the license to require free redistribution, we eliminate the temptation for licensors to throw away many long-term gains to make short-term gains. If we didn't do this, there would be lots of pressure for cooperators to defect.
Slight tangent, but you seem to know about licences... Do you happen to know of a licence that has anything like a "can only be used for the benefit of humanity" clause?
I've favoured the MIT licence for what little OSS I've published thus far. But, I'm becoming increasingly concerned that ruthless profit-above-all-else driven companies can include my (benign) work in systems that causes real harm.
That’s far too subjective to be of any legal value. If you want that, you’ll need to (a) spell out what you want to allow, (b) spell out what you want to disallow, or (c) just write the subjective thing out plain and simple and don’t even bother with complying with license norms (e.g. just write “you can do whatever you want with this provided it is for the benefit of humanity”).
That's a fair criticism. My idea of good is not defined, or static - it adapts over time to the norms and values of society.
Perhaps something like the OpenAI approach to their GPT-3 deal with Microsoft is better. That is, if the work Microsoft do with GPT-3 goes in a direction OpenAI doesn't like, OpenAI reserves the right to veto the work [1].
I think there are some instances where it's definitely bad. E.g. weapons that, if used, can by themselves extinguish humanity. Most instances are not that clear unfortunately - lots of sides to the story, extenuating circumstances, etc. etc.
It's not an easy question. However, as the creator of the software I guess I feel that my opinion should count in how it's used. As a simplistic example, if in some dystopian timeline my OSS were used to facilitate a holocaust I'd like to be able to do something to halt that. It doesn't matter that the perpetrators feel that what they're doing is right.
Of course a person has to have some sort of opinion of it under such conditions, but is it going to come down mostly condemning the weapons enabling the aggressor or thankful for the weapons that enable some measure of violent defense?
The Slaughterbots campaign argued, rightly, I think, that advanced autonomous lethal weapons should be suppressed because they enable unethical uses and unscrupulous actors far more than legitimate defense.
It can't really be seen in isolation from the environment (social, economical, etc) it's going to come into I suppose, but in the real, concrete world we have creating them is not a neutral act, and some of the consequences are reasonably predictable.
>Do you happen to know of a licence that has anything like a "can only be used for the benefit of humanity" clause?
A terrible idea for a number of reasons (in terms of legal enforceability, unintended side effects, and more). The following two articles do a good job of explaining why such a license really isn't practical:
Yes, the unintended side effects of HESSLA (sibling comment) were a surprise for me to read about.
Thank you for the links - I'd not heard of the Hippocratic License but the criticisms are interesting.
Your first assumption is that your inventions are important enough to be of use to “bad people”.
The other is your assumption that you have the objective ability to determine good from bad uses of a benign invention.
I’m increasingly looking for the psychological reasons why these ML models and their outputs cause such an emotional reaction in certain individuals.
For example, the language of opponents of Copilot speaks in absolutes. And when presented with the history of copyright when applied to software the opponents seem to not register that copyright (logically) does not extend to the non-expressive parts of a work.
“In computer programs, concerns for efficiency may limit the possible ways to achieve a particular function, making a particular expression necessary to achieving the idea. In this case, the expression is not protected by copyright."
This allows for verbatim copies if they are utilitarian in nature!
As for why we should allow verbatim copies of utilitarian features... First, let's preface this with the substantial similarity of the structure, sequence and organization as established in Whelan v. Jaslow which amongst other things says that you cannot merely change the variable names if the expressive structure of the code remains the same. Now let's imagine 10,000 software developers who all implement Dijkstra's algorithm in C and then run it through clang-format. Aside from variable names, isn't it safe to assume that many of the implementations are going to be exactly the same?
Now, this doesn’t mean that GitHub is not in violation of other copyright claims, such as clearly expressive parts like comments and more!
Very cool but I don't care about saving humanity either, parent of parent asked a valid question and "FSF says so" with some hand wavy rationale is just not a very satisfying answer.
They call it a "nuclear option" which generally implies some level of effectiveness. The fact that nobody would agree to go along with this sort of scheme renders it ineffective. This isn't a nuclear option, it is a wet fart option.
I'm not trying to be contrarian here, I was curious why not and why this isn't a thing already. I'm just more of a programmer guy and less of a lawyer guy
Then don't use an open source or free software license. Write your own custom license (perhaps consult with legal counsel in the process) and use it for the software you create.
I don't argue that such licenses are bad (though the FSF might), just that they are neither open source nor free.
Oh, yeah well that's not a real open source license. Apologies I read the "open source" more as in "it's on github" and was a little confused what all the organizations and definitions have to do with the actual idea
What a wonderful excuse for a government to pay another 1 billion dollars on crapware to their cronies, who will outsource it to some incompetent software sweatshop on the other side of the world.
We can barely get governments to use open source even today, without restrictions. Hell, we can barely make them manage source code for commercial products they commission and pay for. I've walked into govt shops that were 100% binary dependent to the original software author, which never delivered source code and charged them trough the nose for the basic servicing.
Like it or not, the government and regulators represent us, we need individual accountability but harming the govt. directly harms ourselves firstly. The bureaucrats and the corrupt hardly care.
Companies have a similar problem now with AI than what the music labels had with Napster and MP3s in the 90's. Music labels tried very hard to legislate the problem away but it failed. I remember Metallica's Lars Ulrich working hard to fight it. They finally embraced the change. If it can't be done in the U.S., it will be done in some other country. That country will have the competitive advantage.
We'll go thru the same with AI but ultimately it won't be stopped. As long as there's no world wide coordination limiting its impact, AI will continue its course.
They did legislate the problem away. Sure, Spotify and YouTube play a part in the reduced music piracy today. But it also helps that all of the music piracy sites have been killed, and the only ones left are shady enough that you fear malware if you go there.
There is a load of pirated content on there, but YouTube have successfully cowed every creator with a decent-sized audience into fearing more than a second or two of copyrighted music appearing in their videos.
YouTube didn't get sued early on because it didn't have any money to sue. After it was acquired, it was sued, and the current system is the agreement that Google and the rights holders came to (which has been incredibly profitable for both sides).
Models that are trained on data under open source licenses (such as Creative Commons) would likely be much safer from copyright claims. I like to use the Debian Deep Learning Team's Machine Learning Policy to evaluate the openness of ML work.
Unless they carry with them a library of attributions to every source image, that safety comes mostly from anticipating that authors of CC-licensed works won't be too upset about people using them.
Stability AI is formed with that vision to keep is open-source and accessible to the masses. It's very rare that we might see it becoming a closed source
But it might eventually be neutered with filtering out "problematic" content from the models. Maybe that's NSFW now, but then could easily have busybodies start pressing for "bias" and other topics to be removed.
Someone here put it very well to watch out how the masses would try to censor AI-produced content into oblivion in their futile pursuit of trying to shoot the messenger.
About 150m images have been generated with SD so far. That's already a new large scale training set. Generate and curate before retraining to create a virtuous cycle.
> That somehow international legislation will converge on the strictest possible interpretation of intellectual property, and those models will become illegal by the mere fact they were trained on copyrighted material.
Doesn't this ultimately result in local maxima? All the biases get reinforced and all the novelty (things the system hasn't seen/produced yet) goes away.
A tiny example: Dall-E (and SD) both struggled with eye positioning, for example. Wouldn't training a model on their output then reinforce that particular bias of poorly positioning eyes? Now multiply this by every existing quirk in the models.
I think the savvy media companies realize that we're at the cusp of ai generated media - movies and music included. If we have free/open models trained on the past 100 years of media, they may become obsolete and they will fight this to the death.
Irony is the "NSFW" moral concerns, when the media companies put out such negative and filthy content as it is.
The way Disney is churning out rehashed content for its IP they're obsoleting themselves. When your human content is more predictable and stale than something generated by an AI you should hang your head in shame.
I am more hopeful. Unlike popcon/napster these models aren't directly impacting existing bottom line of any company/organization. Most of the models are trained on opensource / public datasets, so you won't find any company to sponsor the fight against these models. The cost of these models is an issue right now, but Mr. Moore has always handeled that well.
Unfortunately, you’re right. These models are beneficial to large corporations, and they do the most harm to the small individual artists who created the content that made them possible in the first place, so it’s unlikely there will be any serious legal challenges.
I don't think that Moore's law is what's driving down ML compute costs in particular, where there seems to be a lot of innovation going on in terms of hardware architecture and compilers (much of which is proprietary). Even just thinking about memory bandwidth, which historically has scaled much slower than compute: the $/second required to push 10+ TB of training data into some piece of hardware that can do useful work on it isn't going to fall by 100x in a decade.
It is, but 4chan is hardly the mainstream internet, and there's a lot worse than celebrity deepfakes on it. On Reddit, it has been relegated to a few pariah subreddits. Earlier, you would have spotted some on the homepage.
Not sure what you mean by "mainstream internet", it's a normal page anyone can access by typing its address into the browser. Well known, too. And if you're looking for this kind of thing it's the first suggestion you're going to find.
Sure, it's not on the most visited homepages of the world - but it hardly went away. Even on the most visited homepage it's just few clicks away.
What's the point of downloading it when it'd just stagnate? This isn't like regular software where people can easily put in hard work and sweat to improve it.
LLMs have the unfortunate limitation of being both powerful and lending themselves to centralized control choke-points due to how resource intensive they are to train. Under this paradigm, I fear commercial entities will be able to easily navigate the legal landmines and continually improve while open efforts perpetually lag far behind.
There are many vested interests who want this control for various reasons they justify as: protection from x-risk, keeping it out of the hands of abusers and bullies, economic advantage. Their reasons for want of control are either well intended but wrong-headed or profit-motivated and disingenuous.
Rather than challenging the likes of GPT-3 and Copilot enabling freedom, I fear folks will be forced to send all their videos, pictures, text and code to the servers of Microsoft, Amazon and Google or lose access to advantages as LLMs continue to improve at a rapid clip.
> LLMs have the unfortunate limitation of being both powerful and lending themselves to centralized control choke-points
It was hard to accomplish, but you can finetune SD on your computer. They are working on instruction-tuning LLMs as well. In general ML models are not closed boxes inaccessible to us - they can be finetuned, reprompted, you can even average two versions to get a mix of two models. In the last 2 years lots of papers were written on finetuning and prompting, all of them geared towards low resource AI adaptation to new tasks.
But you can't selectively re-train them, can you? As in, don't use elements from this part of the training data anymore, but use elements from this body of work that wasn't part of the training data? If I understand correctly you'd still need a full re-training for that.
- lexical filtering by applying a blacklist of artist names on the original prompt
- perceptual filtering - drop all generated images that look too close to copyrighted images in your training set
- re-captioning based filtering - use a model to generate captions for an image and apply filters on the captions; you can also filter by visual style
- CLIP based filtering where you use embeddings to find nearest neighbours, and if they are copyrighted then you can drop the image
- or train a copyright violation detection model that takes generated images and compares them to images from the original authors
Copyright enforcement struggles are going to be interesting to watch in this decade. But I think it will slowly become irrelevant, because anything can be generate again slightly different until they finally pass the filters.
I was aiming more at the centralized-control angle (though I didn't make that very clear), i.e. are open-source models actually viable long-term? If only orgs with absurd amounts of compute can do updates because those imply a full re-training, wouldn't that effectively centralize control over any such model? Is there the option to to an incremental, limited re-training?
Much of modern deep learning is actually premised on the discovery that training on a large, noisy dataset _first_, and then fine tuning (starting training on new data with the same weights) is generally quicker to converge, and also more accurate.
This is part of the motivation for “foundation models”.
There’s another paradigm called student/teacher models where a randomly initialized model updates it’s weights according to another pretrained model. This could (maybe?) be used to achieve the desired effect of a model that learned in a “clean room”.
That's fine until the next brand new model based on a better architecture where the above hacks won't suffice. My concerns here are long term, like 1 or 2 years out in AI-years.
What about future models with fewer artifacts that are much easier to communicate with and better at generation? Opportunity costs might favor just sending your data to and paying corps with compliance guarantees than spend time fiddling with 2022 era diffusion models. And don't forget this affects 3D, video snippets and music going forward.
> that many companies will, ironically, use "ethical" as a pretext to not be open.
Yes, weaponized ethics as sleight of hand for control is a common historical pattern.
"Opportunity costs might favor just sending your data to and paying corps with compliance guarantees than spend time fiddling with 2022 era diffusion models."
This is exactly why I pay $30 per month for MidJourney. The output is just phenomenally better than most of the images coming out of SD, and the UI is much better as well. It's just not worth my time fiddling with SD if the results are so bad in comparison.
If/when SD catches up, I'd jump ship to using it in a heartbeat.
While it'd be difficult to improve upon the model, it might be easy enough to finetune it if needed, and it's certainly worth it to USE it as is.
There is a limited number of models costing 6 digits in dollars in train time and are freely available. There is certainly value in preserving them, in a world of artificial scarcity.
It's not just processing power that smaller open projects lack in comparison to large corporations, but data.
AI thrives and depends on large amounts of clean, well labeled data.
Large corporations understand this and have hoarded data for a long time now. Some of them have also managed to label this data by millions of people through things like Recaptcha, or just by hiring lots of people to do it.
Open datasets tend to be much smaller and dirtier than small, open projects have access to.
I suppose it would be possible to, over time, collect lots of data and crowd-source some project to clean it up and label it well enough to be useful, then crowd-source the AI model training itself, but it would probably take a long time and by then corporate-owned AI models will already dominate (as they do now with MidJourney, for example, being way better in my experience than Stable Diffusion, but with time the difference will only get starker).
I'd also be concerned with such ostensibly open projects eventually going closed and commercial as IMDB did after getting lots of work by volunteers freely giving their time to writing reviews.
You need to be very careful about making sweeping generalizations based on a single personal anecdote. The really large data sets typically have very high error rates and sample biases. For instance, Google’s JTF300M is far noisier than ImageNet, which itself is hardly free of errors and biases. Any data set with hundreds of millions to billions of images will generally contain a large proportion of images and labels scraped from the web, w/ automatic filtering or pseudolabeling, perhaps w/ some degree of sampled verification by human labelers.
In fact, generally DL is quite tolerant to label noise, especially using modern training methods such as SSL pretraining.
Similar to this approach, at qblocks.cloud we bring under-utilized GPU servers from crypto miners and data centers to use for AI training and deployments at 50-80% low cost than traditional clouds. On-demand and at scale.
It is possible but not practical scaling-factor-wise when synchronization demands, communication bottlenecks on heterogeneous hardware and connection speeds are accounted for. The larger the transformer model, the less practical this quickly becomes.
A fair compromise is any marketplace for clusters with good interconnect but a lot cheaper than the cloud. Tuning distributed training and network transport layer for settings not as homogeneous as the cloud will also help on top of generally good interconnect. Security is a concern.
Building on points raised by pmoriarty, being able to scrape data makes up for lacking labeled data in the era of self-supervised training. IP-hawks are now putting a damper on that option, which is why I worry this might backfire from a freedom perspective.
This is the first time I've heard this idea, but even with all the initial objections, I think this is the future. Something like this is going to happen some day, and I think it'll probably be in the next 5-20 years.
I even think there will be multiple initiatives like this, and there will be at least 1 big repository that accepts inputs and retrains periodically for anyone who wants the model.
What's the point of downloading it when it'd just stagnate?
The quality of the output you can get with the models right now have perpetual utility IMO. If you use it to create patterns, backgrounds, or even just for inspiration creations right now, it might be a shame if it didn't progress (depending on your position) but it's fine as-is if you put in the work to compose and refine the raw output.
The only way to get rid of centralized choke points is to actually go decentralized.
At Q Blocks, we're working on making this solution a reality for a lot of the ML devs constrained by the computing costs on cloud.
>LLMs have the unfortunate limitation of being both powerful and lending themselves to centralized control choke-points due to how resource intensive they are to train.
I wonder if that will continue.
My understanding is that's partially because it currently relies on GPUs, which until relatively recently there was a limited demand for, and the market is basically controlled by a single company.
Will we see cheaper special purpose AI accelerators? Like happened with crypto mining ASICs.
> those models will become illegal by the mere fact they were trained on copyrighted material
The blog post says they are worried about the ability to use the model to "use it for illegal purposes and hurting people". I think that they are referring to the ability to create all kinds of compromising pictures (porn) with celebrities, kids, etc. Am I misreading that? They don't mention copyright anywhere.
> Am I misreading that? They don't mention copyright anywhere.
The conspiracy theorist would say that if you were doing something you shouldn’t, you wouldn’t mention it. Instead, you’d give a more palatable excuse to buy yourself some time while you figure out how to get away (legally) with the thing you shouldn’t be doing.
> That somehow international legislation will converge on the strictest possible interpretation of intellectual property, and those models will become illegal by the mere fact they were trained on copyrighted material.
That's the only possible interpretation, really. AI models algorithmically remix input intellectual property en masse, without any significant amount of human creativity, the only thing copyright law protects. As such, the models themselves are wholly derived works, essentially a compressed and compact representation of the artistic features of the original works.
Legally, a AI model is equivalent to a huge tar.gz of copyrighted thumbnails: very limited fair use applies, only in some countries, and only in certain use contexts that generally don't harm the original author or out-compete them in the market place - the polar opposite of what AI models are.
I forked deepfake a few years ago because it seemed interesting. I didn’t have a spidy sense just thought it would be something interesting to look into. But I forked in GitHub rather than doing a proper clone so now it’s gone.
It reminds me to follow the datahoarder maxim that if you don’t admin then servers, you don’t have the data. So now I clone stuff to a local drive.
“In terms of the ingestion of publicly accessible code, Ochoa said, there may be software license violations but that's probably protected by fair use. While there hasn't been a lot of litigation about that, a number of scholars have taken that position and he said he's inclined to agree.”
It’s very probably fair use under current copyright laws. The things is that the game is changing very rapidly. Right now it’s suffering from criticism in terms of how it affects the society and allows people to generate unwanted images, and merely copyright laws may not be sufficient to protect them. And it has already caught the regulator’s attention so even the law could be rewritten around these models.
At least in the United States there is the jurisprudence of stare decisis. Case law is the only place where the details of how copyright are applied to software are hashed out. I don’t see this changing in the foreseeable future.
> Case law is the only place where the details of how copyright are applied to software are hashed out
Incorrect. In the absence of new legislation case law is how these things get worked out, but new statutes could be passed and could void prior case law.
Well that's a given in a discussion around common law jurisprudence. I still think it is highly unlikely that this changes. This issue doesn't seem to be making or breaking the political careers of the members of legislative branch anytime soon.
Let me expand on this a bit... if you read through the above text (feel free to search for the below terms) of the current laws around copyright you will notice that there is no discussion of:
Funny way to admit that your original statement was flat-out wrong.
There is no separate "civil law jurisprudence" and "common law jurisprudence". Common law by itself is by definition not jurisprudence. Civil law combines aspects of statutory and common law (as one form of precedent) into a single system. Some aspects of common law even make their way into criminal law. Instead of just Googling for buzzwords, learn what they mean before you try to bluster your way through an argument with them.
"it has already caught the regulator’s attention so even the law could be rewritten around these models"
The law moves slow. Even were that to happen eventually, the laws will very likely be challenged in courts, and those will take a while to be resolved.
Finally, even if the US outlaws this, there'll be plenty of other countries where it'll be legal. There's plenty of infringement of US copyright in China, for instance. The same is likely to happen in regards to AI that's illegal in the US but legal elsewhere.
By the time the law catches up, model creation may become so easy to create by individuals instead of just by deep-pocketed corporations, it may be practically impossible to stop.
Both of your sources make the point that the output of such models is separate from the ingestion mentioned in your (carefully selected) quote, and that the legal definition of fair use might well change to preclude such "AI washed" (my term) copying. That's almost the opposite of how you portray the state of legal thought on the matter.
I'm sorry that I was not making points that support your feelings on Copilot. I was purely discussing the legality of the models themselves, which is what the commenter I was responding to was worried about.
But sure, let's talk about outputs as well. From the second source we can see this from Tyler Ochoa:
"If there's only one good way to do it, OK, then that's probably not eligible for copyright. But chances are that there's just a lot of code in [the training data] that has used the same open source solution, and that the output is going to look very similar to that. And that's just copying."
I have seen some probable copyright violations from the output of Copilot, such as comments and some certain structural similarities that might be protected, although it is hard to say. But focus on the first part of what Mr. Ochoa is saying here, which is also laid out in this quote:
“In computer programs, concerns for efficiency may limit the possible ways to achieve a particular function, making a particular expression necessary to achieving the idea. In this case, the expression is not protected by copyright."
This allows for verbatim copies if they are utilitarian in nature!
As for why we should allow verbatim copies of utilitarian features... First, let's preface this with the substantial similarity of the structure, sequence and organization as established in Whelan v. Jaslow which amongst other things says that you cannot merely change the variable names if the expressive structure of the code remains the same. Now let's imagine 10,000 software developers who all implement Dijkstra's algorithm in C and then run it through clang-format. Aside from variable names, isn't it safe to assume that many of the implementations are going to be exactly the same?
As for why it was carefully selected... more often than not when I bring these things up people who feel upset about Copilot go off to cherry-pick some random quote out of context in order to support their upset feelings. Therefore I'm highlighting the important parts as to help people look beyond their upset feelings.
This is a complicated and nuanced matter. Attempting to channel everything through the lens of "this makes me personally feel bad and must be completely wrong" does not help the discourse. It may make you popular to a certain crowd but it might be unpopular to the public at large and it might also be incoherent from a legal standpoint, akin to bashing your head against a wall at a weekly meetup of the local heads-bashing-against-walls club.
There is plenty of room for discussion on what constitutes not only the legal interpretation of fair use and the idea/expression dichotomy but also the bigger picture. The knife always cuts both ways. Would it be acceptable to the open-source community if Microsoft could stop anyone from publishing Dijkstra's algorithm in C# because they wrote it first?
> I'm sorry that I was not making points that support your feelings on Copilot.
That's a very petulant way to defend cherry picking. I wasn't asking you to support one particular view; in fact that's the problem I was identifying. Your sources presented a balanced view, which you misrepresented by citing only the part that supported your own conclusion.
> focus on the first part
No, because the second part matters too. Here's Lemley and Casey again (emphasis mine):
<<<some purposes—say, ... a translation program that produces a translation of an entire copyrighted work—seem more substitutive than transformative, so that if they run afoul of the ever-broadening definition of similarity in music, fair use is unlikely to save them.>>>
Or the Register:
<<<"I actually think there's a decent chance there is a good copyright claim," said Tyler Ochoa
...
the functional nature of the code means that reproducing it in a suggestion may not be seen as particularly transformative, which is one of the criteria for determining fair use>>>
Those are your own sources undermining - if not outright contradicting - your one-sided interpretation.
The limitation to market harms in the four-factor test for fair use should not be considered permanent. Law is, after all, a social construct. There's ample precedent for considering harms to the commons, to communities, and so on in other areas of law. Also, there might indeed be market harms. If a company open-sources some of their code but also hopes to profit by selling it in pre-packaged form or as a service, then AI-washed copying could constitute harm in even the most market-myopic terms. The "transformative" test is also pretty suspect in the context of AI-assisted copying, but this is getting long enough so I'll not go down that rabbit hole just yet.
> <verbosity about "utilitarian" copies which are not the issue here>
Enjoy your red herrings. I don't share your taste for them.
Again, there is a distinction to be made between the outputs of the model and the model itself!
When Tyler Ochoa is saying that there is a decent chance of a copyright claim he is specifically talking about the output of the model.
Here is the full quote:
In the Texas Law Review in March, 2021, Mark Lemley, a Stanford law professor, and Bryan Casey, then a lecturer in law at Stanford, posed a question: "Will copyright law allow robots to learn?" They argue that, at least in the United States, it should.
"[Machine learning] systems should generally be able to use databases for training, whether or not the contents of that database are copyrighted," they wrote, adding that copyright law isn't the right tool to regulate abuses.
But when it comes to the output of these models – the code suggestions automatically made by the likes of Copilot – the potential for the copyright claim proposed by Butterick looks stronger.
"I actually think there's a decent chance there is a good copyright claim," said Tyler Ochoa, a professor in the law department at Santa Clara University in California, in a phone interview with The Register.
The use of the word "but" marks the transition from a discussion around the model itself to the outputs of the model.
Is it not also perfectly clear that Lemley and Casey are also of the opinion that the model itself is fair use?
> Again, there is a distinction to be made between the outputs of the model and the model itself!
Oh, you mean the very first thing I had to explain to you at the beginning of this exchange because you seemed to be ignoring it? Very little of these discussions has been about the models. Most of the discussion is about the outputs, and there the fair-use case is - as Lemley/Casey and Ochoa both concede - much weaker.
But by all means keep going on about feelings. We can all tell it's not others' feelings that are being hurt by mere disagreement.
> From my research the general consensus is that the processing of copyrighted material will be considered fair use. Here is a lengthy legal discussion:
IANAL but I would take any opinions on this right now with a huge grain of salt and treat them more as advocacy than actual predictions of any legal outcomes.
Whether there is a good case for it being considered fair use doesn't matter at all until its actually litigated and historically the result with fair use in relation to new technologies has always been a crapshoot.
The result could easily be affected by the actual cases that get litigated, and one well chosen lawsuit where machine learning software is shown to produce output that's too close to the material it was trained on could result in a completely different outcome.
The big problem is that while under current law it's pretty clear that this stuff should all be fair use, enough people want it not to be that interpretations and/or laws will possibly change. These laws pretty clearly did not anticipate this sort of thing, so it isn't possible to affirmatively say that anything will hold going forward.
Get 200 interested people backing up 1 TB each and you have your 200 TB backup.
With redundancy and error correction data added to the mix, you should be able to lose a certain percentage of participants and still have access to the full, error-free backup.
Yeah, need to write the program and distributed in r/datahoarder
This should be next on my list since my current project depends on SD model and having data backed up gives me confidence that I can get rid of all their stuff if needed.
Hmm, you can just create a torrent out of it. Either as a single file (impractical, but you can just avoid downloading it fully), or chunk it into multiple files.
You don't even need to store it all at once on your computer: stream it and generate checksums on the fly. Then distribute the torrent, and seed sections at a time. It can also be distributed on IPFS.
I've seen a lot of torrents being used for distributing neural network (mostly stable diffusion forks).
Why would you, though? It's just a list of 5B URLs. Some might go down, some new might go up. But it's not like any government body can suddenly take down all photos on the whole internet...
Yesterday I was backing up and old failing HD. I looked at the models I downloaded since 2014 and since I was out of time, I decided to just delete them. But I deleted them with the same thought you just shared: those old models probably don't even exist anymore, they're probably gone. I'm just hoping that time you described isn't happening anytime soon.
>That somehow international legislation will converge on the strictest possible interpretation of intellectual property, and those models will become illegal by the mere fact they were trained on copyrighted material.
Just feels absurd to me because how is this different from any Human artist who you could equally say was "trained" on copyrighted material.
>Get it now when there are trustworthy sources. Once these kinds of things go underground, it gets much harder to get a trustworthy version.
People have already reverse engineered most text2image models and given enough hardware can train their own. There is no need for this hysterical take. As long as the internet exists you will be able to train these models.
The UK and the EU have already made to law that text and data mining is excluded from copyright for non-commercial uses, and the UK has even done so for commercial use cases.
Personally, I think commercial use cases should get license agreements from the authors for their training data, but I think non-commercial exemptions to advance the field of AI makes sense.
Irregardless of what I think though, the UK has set an international precedent, and the EU is apparently discussing about possibly extending it to commercial use cases as well. So there's that.
I agree that it’s a good idea to download everything now and I agree that the legal powers that be will probably soon force it underground - but I’m less certain the driving reason will be copyright / IP. I think it will be reasons similar to what TA hints at. People are (somewhat understandably) upset with certain classes of output the model is capable of generating and a moral panic is likely to ensue that, historically, has won most cases it’s presented itself in.
I figure these tools fall in a similar category to web scraping which is legal. What you can’t do is copy the file. If you can demonstrate that you are modifying the source data then it’s a new work. Style is not protected by copyright as much as famous artists may want.
Where copyright may be applicable is when the models reproduce original art without modification that a reasonable person wouldn’t know the difference.
I think it'll be EU-style privacy regulations that make it illegal to train on the majority of data. Perhaps the requirement to be able to remove a user's impact from an already computed model if they file a right-to-be-forgotten.
Something that would make any non-trivial model a legal nightmare.
We replaced the title, which has a whiff of press release about it, with what appears to be a representative phrase from the article body. If there's a more representative phrase, we can change it again.
Imagine if Photoshop were deemed somehow unsafe without a mechanism to prevent the user from creating NSFW images. The panic around image generation models is absurd.
Nobody making or studying AI wants to hear "your model is being used to generate copious amounts of CSAM". That's basically a death sentence for the technology - even moreso than "your model is just an unattributed search index of stolen art and code". The easiest way to avoid this is to just ensure the model refuses to generate anything NSFW.
It's not a death sentence and journalists will just find a reason to hate on your project for clicks no matter what you do. Also no children, or adults, are sexually abused when an NSFW image is generated.
"no children, or adults, are sexually abused when an NSFW image is generated"
True, but that might not matter to the general public, legislators, or judges.
I just read a new article on the BBC related to this: "Deepfaked: 'They put my face on a porn video'"[1]
The person in question was not physically abused. Only a fake porn video with her face on it was released. But she was still emotionally distraught over it, and said: "You start thinking about your family," she says, holding back tears. "How would they feel if they saw this content?"
This is a real concern to a lot of people, and I doubt they'll be swayed by people pointing out that they weren't physically abused.
So, yeah, there'll likely be a massive backlash against some AI-generated content and the software that generates it, and laws will likely be made against it in some countries.
But in other countries and on the dark web it'll still exist. There's ultimately no way of stopping it, and eventually people will come to terms with its existence and widespread availability, no matter what the law of some countries says.
So you want it to be open source, but not too open, because then bad people will use it. Good luck with that. If you want to filter everything behind a SaaS like OpenAI go ahead, but then you can't call it open source. And maybe that would have been the right choice. But Pandora's box is open now.
Exactly, that cat is out of the bag. Right now the hard part is not using the models but creating the models. It requires a significant commitment in resources and there are only a handful of companies with those resources. And you need some skilled people to babysit the software and the algorithms.
However, that will inevitably spread to include more and more companies and will also start happening outside the US. All the research around this is being published and there's a lot of open source code that facilitates this. So, it's just a matter of people optimizing and improving that and hardware getting cheaper.
I expect that once that market is big enough, you'll see cloud providers step up with provisioning infrastructure for this stuff. It will still be expensive to use but it won't have a lot of limitations.
AI driven porn is basically the obvious use-case where there are some big companies with lots of money operating in that space and plenty of incentive to make this happen. Morally that might actually be preferable to exploiting people as is their current way of operating. The likes of OpenAI won't be able to do much to stop that.
Powerful people are pulling strings to control AI everywhere. OpenAI is exactly the opposite of open. Now someone is pushing on Stability AI to close it up, I believe those models are more powerful or dangerous than they seem, and it got some people scared in some way.
I read than when some guys from 4chan started running the leaked NovelAI model, they generated porn non-stop for 20 hs or more, no sleep, no eating.
Even without conspiracy theories, these models cost upto 10s of millions to generate, no suprise investors wouldn't like if you are giving it all for free, there should be some revenue model.
To be fair, that's the novelty factor and when this hits "the public" it is not unthinkable that there'll be some "productivity issues".
IMO it is like finding a computer in a world without them. It is mind-blowing and it will take over your mind if you let it. For some folks that results in lots of porn, for others it'll be fear. My guess is that it'll wear off eventually.
> We are forming an open source committee to decide on major issues like cleaning data, NSFW policies and formal guidelines for model release.
I don't see how NSFW photos can easily be stopped from being generated, with the model being open source. Maybe the model could be heavily pre-filtered to remove any photos that could possibly be used for NSFW images.
Which has a LOT of NSFW images in it. I suspect if you removed them from the training set it would go a long way to curb NSFW output but as you say people could easily train their own NSFW latent diffusion model.
My experience with Stable Diffusion is that it has a habit of tripping the NSFW filter - as in, it actually generated NSFW images - on prompts that were entirely innocuous. Would not be surprised if Stability has a huge "how do we get it to stop spitting out porn so easily" problem.
While they frame the post as if this is a positive and something they want to do, reading between the lines, it sounds to me like something has them rattled.
They mentioned regulators here, and I would be curious to hear the story behind that.
Don’t want to go too tin foil hat, but it makes you wonder if a certain other AI company that claims to be “open” may be afraid of a company that actually is open and is applying political pressure.
Oh that's a certainty. They said regulators twice. It was no accident, they are telegraphing just how hard they got smacked behind the scenes.
Extremely likely that the FAANG lobbyists went into overdrive. The big guys know this will be an extremely important industry for the coming decades and don't want a new competitor swooping in with nothing to lose when established companies are forced to be cautious.
I wouldn't call Stable Diffusion "Open Source AI", the training data isn't publicly released under open source licenses. I like the Debian Deep Learning Team's Machine Learning policy for evaluating these things:
His answers on reddit are downvoted and the redditors are correctly pointing out that most of these "protections" smack of the fact that his investors want to stop giving things away and to close up source / resources for better monetization strategies.
I think Daniel Jeffries believes everything they just wrote.
Their new handlers can do anything to the contrary and are incentivized to curb release as well. The market is saying their new handlers are going to do that.
Have to find a way for content makers to make money/jobs through the system. Google search solved that by providing ad revenue to content makers, or else they 'd have removed all their content by now.
download while you can. i really hope this isn’t the beginning of the end for stable diffusion or true open ai. it’s too good to not piss off powerful people. we must keep real open source ai alive, otherwise it’ll only be billionaires like zuck and elon force-feeding us poisonous saccharine.
This appears to be BOTH an IPR statement and a social policy statement.
I tend to thinking they are co-joined, but clarity helps.
I think the social harms side, they need to be careful to under-promise and over-deliver. The likelihood of preventing social harms is frankly close to zero, what they can do is make it more complicated.
Think like this: use stable diffusion to make one "actor" dance a lambada in the left field and save it. in a new state, make a different "actor" dance a lambada in the "right" field. Now using alpha masks combine the two actors. Can this represent sexy dancing? you bet your sweet bippy.
Promising not to release "two person sexy dancing" in this situation would be over-promising. Sure, it was done outside of the AI by masks. Will the law makers care?
(for actor and lambada and sexy dance, substitute whatever contextually means "harm" in a two-actor situation, semantically)
So if someone buys the rights to an artists work and that artist is dead can they start using Stable Diffusion to create new works of art they can claim as "by the artist"?
You can claim it but the emperor is naked, except if the artist actually made generative models that you can run, then the model can produce new art - but I feel it still would have been made by the original creator, not by whoever buys the model.
Hollywood is about to start buying actor image rights and performance data to continue producing movies staring them after death so very likely legislation will be made that does make these things part of the artist canon.
355 comments
[ 4.7 ms ] story [ 267 ms ] thread"What we do need to do is listen to society as a whole, listen to regulators, listen to the community."
"So when Stability AI says we have to slow down just a little it's because if we don't deal with very reasonable feedback from society and our own communities then there is a chance open source AI simply won't exist and nobody will be able to release powerful models."
Looks like someone is leaning on them :(
IMO the AGPL goes a long way to solving that problem. But if the AGPL is not for you, I suppose you could use some non-commercial license terms. It seems like "closed source" is a much better fit for folks who want a great deal of control over licensees. In practice, "closed source" code can be published for licensees to see but instead of granting terms to all comers you could force people to ask you for a license, review their use case and only then decide to grant a copyright license -- with or without source.
Then Amazon comes in and sets up their own even easier hosting while providing none of the profits or development effort back to the original project. The AGPL is no problem because Amazon is fine to publish any changes they make, thats not an issue because the real product is the hosting and not the code.
The problem is releasing stuff as open source and expecting to profit off it while preventing others from doing the same. You fundamentally can't. Why bother with open source with an attitude like that?
AGPL is for code, not for infrastructure, and you can still profit off of AGPL-licensed code by:
1 - selling support for it
2 - getting donations
3 - asking for money in return for working on feature requests
4 - being paid by a corporations to work on an open source product they themselves use and profit from
There's nothing particularly "modern" about this "problem". Redhat caused this problem with GPL and BSD-licensed code in 1993.
It's a feature, not a bug. Some people probably didn't like it then either.
If you resent companies and people making money from your work, you're not really embracing the ideals of open source. So just keep it closed source. Like I said earlier you can always reveal the source to licensees when they pay you, and you can prevent them from distributing the source.
The AGPL is easy to comply with in a way that still earns a company profit while the software authors get nothing.
Agreed that non-commercial public source or just closed source is what these people should be doing.
After taking $100M in venture capital and two distinct drama events due to disorganization, this is unlikely to last.
2) StabilityAI gave RunwayML compute time for them to train Stable Diffusion (they're also the creators of the original model). It's weird to categorize them as " other groups leak the model". They're the ones that created the model! (Source: https://huggingface.co/runwayml/stable-diffusion-v1-5/discus...)
1. The web UIs I have used are taking advantage of the same mental pathways as an electronic slot machine. Just like you can max out your bet on a slot machine and mash a button until you run out of credits, you can do the same on the hosted stable diffusion apps until you get a shareable hit.
2. Just like the dream you had last night, nobody wants to hear about it at breakfast, no matter how epic it was, because it's not backed by any meaning.
That said, I love stable diffusion and am an addict to it almost every day.
https://github.com/AUTOMATIC1111/stable-diffusion-webui/
It is extremely active - author updates it 10-20 times per day.
Have you monitored its temperature while using it? Did your warranty cover it?
Well as long as they have a recent-ish NVidia card, rip AMD users.
I do agree that most of the dross I create is only of interest to me. OTOH I got some laughs with my "Liz Truss holding up signs with rude words on them" series from British friends yesterday.
Who stays awake at night wondering what meaning that cool picture of a dragon had?
They just enjoy it and move on with their day.
Same with AI-generated images... they just look cool, amazing, or hot.. whatever. Most people just enjoy them, without wondering what deep meaning they might have.
It's mostly art critics and the like who wonder what deep meaning Van Gogh's Starry Night or the Mona Lisa have (never mind abstract work like Jackson Pollock's). Everyone else just likes it or they don't.
https://reddit.com/r/StableDiffusion/comments/y9ga5s/stabili...
His comments regarding RunwayML’s release of 1.5 were especially interesting:
> “No they did not. They supplied a single researcher, no data, not compute and none of the other reseachers. So it’s a nice thing to claim now but it’s basically BS. They also spoke to me on the phone, said they agreed about the bigger picture and then cut off communications and turned around and did the exact opposite which is negotiating in bad faith.”
> “I’m saying they are bad faith actors who agreed to one thing, didn’t get the consent of other researchers who worked hard on the project and then turned around and did something else.”
>We want to crush any chance of CP. If folks use it for that entire generative AI space will go radioactive and yes there are some things that can be done to make it much much harder for folks to abuse and we are working with THORN and others right now to make it a reality.
It's a new form of deplatforming - just as people have made careers out of trying to get speech/expression that they dislike removed from the internet, now we're going to see AI companies cripple their own models to ensure that they can't be used to produce speech/expression that are disfavored, out of fear of the reputational consequences of not doing so.
Because there's no evidence it works and the idea makes no fucking sense. It approaches the problem in a way that all experts agree is wrong.
Experts in what exactly?
There are two ways to defend a law that penalizes virtual child pornography:
- On evidence that there is harm.
- On general moral terms, aka "we just don't like that this is happening".
Worth noting that a ban on generated CSAM images was struck down as unconstitutional in Ashcroft_v._Free_Speech_Coalition.
https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...
Now that we can readily generate photorealistic CSAM, there's little to no risk of inadvertently creating an customer base for actual CSAM.
I mean, some SD applications like the interior designer market this as a great tool for potential buyers to try out ideas before they buy.
I'm absolutely certain Linux has been used to kill children. Detergent, pesticide, and even pillows too!
Tools shouldn't be limited based on what the worst way they can be used. Stable diffusion is an absolute positive for society. Even when it's used to generate CP, every image that model creates is not one that involves a real kid.
The cat is out of the bag when OpenAI announced DALLE. Stable Diffusion only accelerated it a bit. Even if Sablity or lawmakers manage to prevent or outlaw open models, criminals will continue to build their own and ignore laws.
The only thing their reluctance does is harm Stability and give their competition a chance to catch up. Perhaps that's a good thing. Maybe it's time for another organization to take the lead.
Many products benefit pedophiles in one way or another. Mobile phones, computers, video editing software, cars (vans?).
"But that's ridiculous, of course we can't prohibit cars just because they can be used by criminals."
Exactly.
More specifically, researchers and hobbyists would be made criminals in this hypothetical. Why would I stop playing with ML just because someone passed a useless and poorly thought out law?
I get the dilemma as a creator though. I wouldn't want my products to be used that way too.
I would add though - if Moore's law continues this will be almost unstoppable in a decade or too.
At that point, I assume RunwayML too talked about releasing it.
Then, a few months later they released it.
And suddenly the response is "How dare they"?
My very rough take on the situation: the company gained their notoriety by building on OpenAI's pioneering research but with an important twist of releasing their models as unneutered open source. Now, their openness is starting to falter due to strong pressure from outside forces.
If they're unable to continue playing the hardball game they themselves invented, I think their glory days will end as fast as they started. The competitive advantage was always their boldness. If they lose that, quickly others will take their place.
In general, I don't think tech that's as open, powerful and easily reproducible as these language models can be stopped. Sure, maybe regulations will delay it a bit, but give it a few years and any decent hacker or tinkerer will be dabbling with 5x better tech with 5x less effort.
[1] https://archive.ph/Z5sU3
You can't predict future legislation. Intellectual property legislation (which is an absolute cancer IMO) can outlaw models and their results. It can outlaw distribution of the data sets, the training, the models. Tech companies already acted way beyond the requirements of the law and effectively censured open source projects like popcorn time.
Can they prevent a determined hacker which already got everything? No.
Could this be the last model to be trained on a wide dataset, available to the public? Yes.
Could they make it a living hell where getting these tools in the future will only be from untrustworthy websites where half the download buttons give you an exe, and all your less tech savvy friends won't bother? Easily.
Could this tool become impossible for companies to use without risking litigation? Very easily.
People tend to forget those making the rules do not have their interests at heart, and every single intellectual property law is designed to leave companies and not people holding all the rights. And those laws can absolutely do damage. Do not underestimate the power of legislation.
At least with copyright law, there's an argument for training being fair use. If generative art becomes a notorious market for CSAM, everyone in the field goes to jail.
[0] Also, I'd like to know what your opinion is on GitHub Copilot. A lot of people decry Copilot for stealing code but love Stable Diffusion for being public, even though they're the same concept and trained in the same quasi-ethical way.
[1] https://www.reddit.com/r/StableDiffusion/comments/y9ga5s/com...
If you constantly watch videos of people eating cheeseburgers, you might want to eat a cheeseburger yourself.
More likely people will just generate more synthetic content to consume.
But they are not harming anyone else.
Hasn't this nonsense been thoroughly debunked by multiple studies at this point? I would assume evidence and "modern thinking" supports the exact opposite of what you claim, unless by modern thinking you mean the same thinking that tries to hide research they don't like.
Video games do not cause violence. End of story.
> I would not assume that the reactions to simulated sexuality is the same as simulated violence.
A would not assume anything. Conduct research and draw conclusions. Don't speculate.
This is the right conclusion, but the logic is entirely wrong.
The reason why video games do not cause violence is that play violence is not anywhere close to the real thing, not that people firewall off fiction from reality. There's plenty of cases in which a piece of fiction has changed people's views! Crime shows are notorious for skewing how actual juries rule on cases. Perry Mason[0] taught them to expect dramatic confessions and CSI[1] taught them to weigh whiz-bang forensics over other kinds of evidence.
In the specific case of porn, there isn't really a difference between "play sex" and "real sex": they poke the same regions of your brain. And the people who are responsible for keeping actual pedophiles from reoffending are pretty much unanimous that the worst thing you can do is give them a bunch of, uh... let's call it "material". So if you're already a pedophile, giving you access to simulated CSAM won't substitute for the real thing. It'll just desensitize you to reoffending.
[0] https://en.wikipedia.org/wiki/Perry_Mason_syndrome
[1] https://en.wikipedia.org/wiki/CSI_effect
>> Conduct research and draw conclusions. Don't speculate.
this is retarded, if you watch a movie, play a video game, read a book with crime events then you will become a criminal. We have a ton of shooter games and still no evidence that this caused more gun violence around the world.
There are way too many factors at play to simply point at porn, which is probably harder to obtain now in all honesty. I found many random porn magazines/pages as a child. Never did I ever go looking for it, but finding it was always a thrill.
People buy less magazines now (based on convenience store shelves increasingly excluding them.)
Same with CP. You have to be sick to enjoy it. Very sick.
It's not well explored at all and you just made that up lmao.
That's akin to the idiotic arguments of the past that "allowing people to see homosexuality will make them homosexual!"
Completely ridiculous.
I'm familiar with the research in this area, and that's not something you can say confidently; most work (and by that I mean 2 or 3 papers in total) has gone into investigating the role 'generated' depictions of CSAM play in the collections of hoarders. No psychological study, as far as I'm aware, has conducted an investigation on those who enjoy cartoon material akin to what you might find in a Japanese manga.
In fact, there's some evidence against what you're saying; anthropological research on fans of cartoon material ('lolicons' or 'shotacons') in Japan shows that their communities draw hard lines between '2D' and '3D' not just in this area of sexuality, but in their sexualities as a whole. This sexual inclination toward the 2D world is termed the 2D-complex and is akin to 'digital sexuality' or fictophilia, not pedophilia.
By way of analogy, perhaps BDSM would work as a good counter point to you. Many people (some studies suggest the majority of people) engage in 'rape fantasies' or other such fantasies of illegal or immoral nature, yet although actual depiction of rape is rightly banned by the state, its simulated variants are not, and we are comfortable to acknowledge that sexual desires do not always manifest in real life, and sometimes the thrill of fantasy itself is the attraction. To make it real would, ironically, defeat the whole point.
One example of a bad thing - you could easily imagine an instagram bot that looks for pictures of people with their kids, then uses a Stable Diffusion like model to produce pictures of the people having sex with their kids, or horrible things happening to the kids, and reply to the target account. The bot might threaten to post the pictures and accuse the person of being a pedophile unless the person pays X in bitcoin (or whatever). Or, the bot could just post such pictures for fun.
I think we don't know if fake CSAM will have a good or bad effect on pedophiles and, sadly, there is no real way to reliably test that (so far as I know). Fake CSAM might placate pedophiles, or it might whet their appetite. It's hard to know what to do.
I think we will eventually get to the point where very good unrestricted image generation models are available to the general public. When that happens there will be chaos - you will live to see man-made horrors beyond your comprehension.
And of course, I'm not looking forward to the world ushered in by free roam with this technology, mainly for the reasons you stated.
I’m referring to legally, sorry I should have specified.
[0] Evidence Mounts: More Porn, Less Sexual Assault. https://www.psychologytoday.com/us/blog/all-about-sex/201601...
Detection becomes easier - is it pornography with a child in it?
Generation starts to become trivial - this video, but this person has the features of an X year old.
At least in the latter case no-one's actually getting raped.
No one will go to jail, except maybe some people who get caught creating, distributing or collecting those images.
A human artist is obviously capable of generating CSAM, even if they have never seen that before.
Filtering of training data is countered by increasing capabilities to generalize:
Two years ago, that was a viable strategy: models could barely produce what was in the training data again.
Today models can generalize much better and compose concepts they have been trained on into new concepts that they haven’t.
Two years from now, filtering will be irrelevant.
Keep in mind, there already are a lot of illustrated/anime style pictures of CSAM on that site of years though (something that is legal in many countries), so it's sort of becoming a blurred area as these AI art generators are still somewhat like that but now are getting to be more photorealistic.
As far as the models not being trained on NSFW content, there was already leaked models that were, and there are unofficial models trained by outsiders using SD that are specifically trained on for example adult image websites.
Then there are narratives. They are weaved so that the suggested actions and solutions will somehow fit the interests of the participants. The narrative can be CSAM, it can be copyright of artists and owners of the training set, the narrative can be disinformation. The narrative doesn't care that current laws do not prohibit anything and that it's all legal. The narrative justifies actions the participants wanted to do because of their interests.
And finally there are actions. They can push legislations, but that's not the only tool (and yes it's slow). Companies can always comply and cooperate, especially when their interests align. Google itself is a participant, with Imagen. They can create a restrictive policy and kick things off their search engine, because that is in their interests too, not because of a narrative or legislation. Just like they profited in YouTube for every piracy site suppressed.
The interests of every single company is stacked against individuals running this at home for free. There are enough narratives to be weaved to justify actions which would stop that.
For decades, and in many countries even today, just getting paid to drive someone in your car is illegal, and you need a "taxi license". It doesn't need to make sense. We could end up with required license to use generative AI in 10 years and nobody would bat an eye after decades of propaganda and narratives.
https://youtu.be/0_BBRNYInx8?t=85 This video (released yesterday) talks about how SD takes an image, converts it to latents.
You'd need to decode those latents back to an image representation and scan. (possibly other ways but that's the most straightforward I can come up with although time intensive).
The entire art tradition is based on copying for study, and then using our brains to convince ourselves that we've "transformed it enough" or "my reference is obscure enough that no one will know."
Now we've simplified that process, and more people are exposed to the risk. I hope that the law takes a minute to evaluate the pace of change instead of saying "ITS TOO DANGEROUS, we must BAN IT", but my hopes are low.
> getting these tools in the future will only be from untrustworthy websites where half the download buttons give you an exe
These models can already be downloaded via well known (ie community reviewed) torrents. So can many terabytes of labeled training data. This particular horse is well out of the barn.
Somewhat similar gut feeling to when popcorn time was released, although it might not be exactly the same.
While I really wish I'm wrong, my gut tells me that broadly trained machine learning models available to the general public won't last and that intellectual property hawks are going to one day cancel and remove these models and code from all convenient access channels.
That somehow international legislation will converge on the strictest possible interpretation of intellectual property, and those models will become illegal by the mere fact they were trained on copyrighted material.
So reminder to everyone: Download! Get it and use it before they try to close the Stable doors after the horses Diffused. Do not be fooled by the illusion that just because it's open source it will be there forever! Popcorn time lost a similar battle.
Get it now when there are trustworthy sources. Once these kinds of things go underground, it gets much harder to get a trustworthy version.
[1]: https://huggingface.co/runwayml/stable-diffusion-v1-5
0: magnet:?xt=urn:btih:3a4a612d75ed088ea542acac52f9f45987488d1c&dn=sd-v1-4.ckpt&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a6969%2fannounce&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337
So a paywall. I agree it's a annoyingly misleading term for the concept and would be happy to hear better alternatives[0], but I haven't found one yet.
0: like eg "passphrase" instead of "password" or "assuming the conclusion" instead of "begging the question" for their respective concepts
Seconded, actually. I do have a bad habit of assuming people already know this.
> Official sources are recommended for this reason.
Very not seconded; see for example comments elsewhere in this thread about untrustworthy sources for popcorn time, and recall that the GP was specifically discussing the risk of Stability AI deciding to kill this.
You mention Popcorn time. I wonder if torrents in general could be a great example of how something like this plays out? Torrenting took the world by storm and had an amazing "product-market fit" for the early internet days. Of course, downloading copyrighted material was always illegal but that didn't stop many.
Over time, legal but paid alternatives rose up: Spotify, iTunes, Netflix. These players found their place in the market by balancing the interest of copyright holders and the needs of users looking for cheap and easy access to entertainment.
Just as Netflix acquired large content libraries, same here. With enough money, large training datasets could be acquired in a legally solid manner.
It's interesting to think where this analogy might fail as well, and how the paths of these technologies could differ. For one, torrenting was mostly for entertainment, and thus impacted B2C first. On the other hand, language models are more so for media _creation_ and the B2B sphere.
Like torrents, you first have to resort to random websites who get randomly taken down as they acquire reputation. If a person takes the face and responsibility for something, he gets litigated into oblivion.
So you get to the point where trustworthy and untrustworthy sources are indistinguishable
. Now what they do is create untrustworthy sources. Like time for popcorn. Sow discord.
Fork several times, create intentionally malwared versions of both the program and the website. Keep kicking off the trustworthy sources of search engines, while magically skipping takedown requests for the less trustworthy websites.
Find ways to break old versions if possible, just to force them to keep moving. (they can make gradio randomly change APIs just to break the old trustworthy versions)
All of this can happen.
Before this gets flagged to oblivion, this is obvious. You just have to recognise that the "regulators" and industry insiders Emad is trying to "shield" you from are enemies and ask yourself, how do I hurt them?
And reasonably technical people have zero issues, as it should be.
Media piracy has been abundant my entire life. It's never slowed down or become inaccessable.
You didn't mention one of the largest (perhaps even the largest) distributor of copyright content (which happens to also be free, for now): YouTube.
You can watch/listen to endless amounts of copyrighted content (and other types of content) on there completely for free, and to say it's tremendously popular would be an understatement.
Google has made it work through ads. Perhaps something like that will happen with image-generating AI.
Tit-for-tat. Regulators and artists don't want this? Okay, include in all open source software licenses that regulators and artists are now barred from using them without payment.
Also, discriminating like you suggest would make those licenses closed source by definition.
Long live clopen source!
Definitions are not what matters in the end. Why doesn't the viral and restrictive element of GNU's GPL license make the license "non-open"?
"Open source" has a definition (https://opensource.org/osd) and the GPL meets it, because it doesn't prevent derived worries from being distributed under the same license.
https://www.gnu.org/licenses/license-list.html
> [...] is a nonfree license because it extends the four freedoms only to some kinds of organizations, not to all. Such a restriction in a software license, in the name of any cause whatsoever, imposes too much power over users. Please don't use this license, and we urge you to avoid any software that has been released under it.
https://opensource.org/osd-annotated
> 1. Free Redistribution
> The license shall not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources. The license shall not require a royalty or other fee for such sale.
> Rationale: By constraining the license to require free redistribution, we eliminate the temptation for licensors to throw away many long-term gains to make short-term gains. If we didn't do this, there would be lots of pressure for cooperators to defect.
I've favoured the MIT licence for what little OSS I've published thus far. But, I'm becoming increasingly concerned that ruthless profit-above-all-else driven companies can include my (benign) work in systems that causes real harm.
Perhaps something like the OpenAI approach to their GPT-3 deal with Microsoft is better. That is, if the work Microsoft do with GPT-3 goes in a direction OpenAI doesn't like, OpenAI reserves the right to veto the work [1].
[1]: https://www.ted.com/talks/the_ted_interview_the_race_to_buil...
It's not an easy question. However, as the creator of the software I guess I feel that my opinion should count in how it's used. As a simplistic example, if in some dystopian timeline my OSS were used to facilitate a holocaust I'd like to be able to do something to halt that. It doesn't matter that the perpetrators feel that what they're doing is right.
The Slaughterbots campaign argued, rightly, I think, that advanced autonomous lethal weapons should be suppressed because they enable unethical uses and unscrupulous actors far more than legitimate defense.
It can't really be seen in isolation from the environment (social, economical, etc) it's going to come into I suppose, but in the real, concrete world we have creating them is not a neutral act, and some of the consequences are reasonably predictable.
[1]: https://www.gnu.org/licenses/hessla.en.html
>The Software shall be used for Good, not Evil.
AIUI, it was put it mostly as a joke.
https://www.cnet.com/culture/dont-be-evil-google-spurns-no-e...
A terrible idea for a number of reasons (in terms of legal enforceability, unintended side effects, and more). The following two articles do a good job of explaining why such a license really isn't practical:
https://perens.com/2019/09/23/sorry-ms-ehmke-the-hippocratic...
https://www.gnu.org/philosophy/programs-must-not-limit-freed...
Your first assumption is that your inventions are important enough to be of use to “bad people”.
The other is your assumption that you have the objective ability to determine good from bad uses of a benign invention.
I’m increasingly looking for the psychological reasons why these ML models and their outputs cause such an emotional reaction in certain individuals.
For example, the language of opponents of Copilot speaks in absolutes. And when presented with the history of copyright when applied to software the opponents seem to not register that copyright (logically) does not extend to the non-expressive parts of a work.
“In computer programs, concerns for efficiency may limit the possible ways to achieve a particular function, making a particular expression necessary to achieving the idea. In this case, the expression is not protected by copyright."
https://en.wikipedia.org/wiki/Abstraction-Filtration-Compari...
This allows for verbatim copies if they are utilitarian in nature!
As for why we should allow verbatim copies of utilitarian features... First, let's preface this with the substantial similarity of the structure, sequence and organization as established in Whelan v. Jaslow which amongst other things says that you cannot merely change the variable names if the expressive structure of the code remains the same. Now let's imagine 10,000 software developers who all implement Dijkstra's algorithm in C and then run it through clang-format. Aside from variable names, isn't it safe to assume that many of the implementations are going to be exactly the same?
Now, this doesn’t mean that GitHub is not in violation of other copyright claims, such as clearly expressive parts like comments and more!
I'm not trying to be contrarian here, I was curious why not and why this isn't a thing already. I'm just more of a programmer guy and less of a lawyer guy
I don't argue that such licenses are bad (though the FSF might), just that they are neither open source nor free.
We can barely get governments to use open source even today, without restrictions. Hell, we can barely make them manage source code for commercial products they commission and pay for. I've walked into govt shops that were 100% binary dependent to the original software author, which never delivered source code and charged them trough the nose for the basic servicing.
Like it or not, the government and regulators represent us, we need individual accountability but harming the govt. directly harms ourselves firstly. The bureaucrats and the corrupt hardly care.
We'll go thru the same with AI but ultimately it won't be stopped. As long as there's no world wide coordination limiting its impact, AI will continue its course.
The iPod would not have had the impact it had without piracy.
Is it legal? I don't know. I guess they have the fig-leaf of taking down copyrighted content when asked.
Fig-leaf not withstanding, if Google (YouTube's owner) didn't have such deep pockets I'd be amazed they didn't get sued in to oblivion like Napster.
Countries that don't do that will be just as successful in the world marketplace as are countries that don't respect copyright.
https://salsa.debian.org/deeplearning-team/ml-policy
Doesn't this ultimately result in local maxima? All the biases get reinforced and all the novelty (things the system hasn't seen/produced yet) goes away.
A tiny example: Dall-E (and SD) both struggled with eye positioning, for example. Wouldn't training a model on their output then reinforce that particular bias of poorly positioning eyes? Now multiply this by every existing quirk in the models.
Irony is the "NSFW" moral concerns, when the media companies put out such negative and filthy content as it is.
What I would value much more is the writing, directing, editing, and acting... and you can't yet get very good quality of any of that through AI yet.
Maybe someday, but not today.
There is a legal machinery that works behind the scenes which we aren't always aware of.
Sure, it's not on the most visited homepages of the world - but it hardly went away. Even on the most visited homepage it's just few clicks away.
LLMs have the unfortunate limitation of being both powerful and lending themselves to centralized control choke-points due to how resource intensive they are to train. Under this paradigm, I fear commercial entities will be able to easily navigate the legal landmines and continually improve while open efforts perpetually lag far behind.
There are many vested interests who want this control for various reasons they justify as: protection from x-risk, keeping it out of the hands of abusers and bullies, economic advantage. Their reasons for want of control are either well intended but wrong-headed or profit-motivated and disingenuous.
Rather than challenging the likes of GPT-3 and Copilot enabling freedom, I fear folks will be forced to send all their videos, pictures, text and code to the servers of Microsoft, Amazon and Google or lose access to advantages as LLMs continue to improve at a rapid clip.
It was hard to accomplish, but you can finetune SD on your computer. They are working on instruction-tuning LLMs as well. In general ML models are not closed boxes inaccessible to us - they can be finetuned, reprompted, you can even average two versions to get a mix of two models. In the last 2 years lots of papers were written on finetuning and prompting, all of them geared towards low resource AI adaptation to new tasks.
- lexical filtering by applying a blacklist of artist names on the original prompt
- perceptual filtering - drop all generated images that look too close to copyrighted images in your training set
- re-captioning based filtering - use a model to generate captions for an image and apply filters on the captions; you can also filter by visual style
- CLIP based filtering where you use embeddings to find nearest neighbours, and if they are copyrighted then you can drop the image
- or train a copyright violation detection model that takes generated images and compares them to images from the original authors
Copyright enforcement struggles are going to be interesting to watch in this decade. But I think it will slowly become irrelevant, because anything can be generate again slightly different until they finally pass the filters.
This is part of the motivation for “foundation models”.
There’s another paradigm called student/teacher models where a randomly initialized model updates it’s weights according to another pretrained model. This could (maybe?) be used to achieve the desired effect of a model that learned in a “clean room”.
Because it's already good enough to have made it's way into many of my workflows.
I do feel that many companies will, ironically, use "ethical" as a pretext to not be open.
I mean this isn't even speculative anymore after what happened with - hilariously named - OpenAI
> that many companies will, ironically, use "ethical" as a pretext to not be open.
Yes, weaponized ethics as sleight of hand for control is a common historical pattern.
This is exactly why I pay $30 per month for MidJourney. The output is just phenomenally better than most of the images coming out of SD, and the UI is much better as well. It's just not worth my time fiddling with SD if the results are so bad in comparison.
If/when SD catches up, I'd jump ship to using it in a heartbeat.
While it'd be difficult to improve upon the model, it might be easy enough to finetune it if needed, and it's certainly worth it to USE it as is.
There is a limited number of models costing 6 digits in dollars in train time and are freely available. There is certainly value in preserving them, in a world of artificial scarcity.
Is it possible to crowdsource AI training with something that looks similar to folding@home?
AI thrives and depends on large amounts of clean, well labeled data.
Large corporations understand this and have hoarded data for a long time now. Some of them have also managed to label this data by millions of people through things like Recaptcha, or just by hiring lots of people to do it.
Open datasets tend to be much smaller and dirtier than small, open projects have access to.
I suppose it would be possible to, over time, collect lots of data and crowd-source some project to clean it up and label it well enough to be useful, then crowd-source the AI model training itself, but it would probably take a long time and by then corporate-owned AI models will already dominate (as they do now with MidJourney, for example, being way better in my experience than Stable Diffusion, but with time the difference will only get starker).
I'd also be concerned with such ostensibly open projects eventually going closed and commercial as IMDB did after getting lots of work by volunteers freely giving their time to writing reviews.
More recently the open LAION data sets have become widely used by both tech giants and independent researchers.
The problem is DL is really sensitive to dirty data, disproportionately so.
At $DAYJOB once we cleaned the dataset, removed a few mislabeled identity/face pairs (very few, about 1 in 1e4) and the metrics goes up a lot.
In fact, generally DL is quite tolerant to label noise, especially using modern training methods such as SSL pretraining.
https://arxiv.org/pdf/1705.10694.pdf https://proceedings.neurips.cc/paper/2018/file/a19744e268754... https://proceedings.mlr.press/v97/hendrycks19a.html
A fair compromise is any marketplace for clusters with good interconnect but a lot cheaper than the cloud. Tuning distributed training and network transport layer for settings not as homogeneous as the cloud will also help on top of generally good interconnect. Security is a concern.
Building on points raised by pmoriarty, being able to scrape data makes up for lacking labeled data in the era of self-supervised training. IP-hawks are now putting a damper on that option, which is why I worry this might backfire from a freedom perspective.
I even think there will be multiple initiatives like this, and there will be at least 1 big repository that accepts inputs and retrains periodically for anyone who wants the model.
The quality of the output you can get with the models right now have perpetual utility IMO. If you use it to create patterns, backgrounds, or even just for inspiration creations right now, it might be a shame if it didn't progress (depending on your position) but it's fine as-is if you put in the work to compose and refine the raw output.
I wonder if that will continue.
My understanding is that's partially because it currently relies on GPUs, which until relatively recently there was a limited demand for, and the market is basically controlled by a single company.
Will we see cheaper special purpose AI accelerators? Like happened with crypto mining ASICs.
i was actively following torrentfreak at the time and there was genuine excitement with something incredible but that only lasted a week :-(
why do you say they lost the battle? the original team threw in the towel within the week but there are people who have taken the fight
https://github.com/popcorn-official/popcorn-desktop/releases... here, the latest release was on 04 Sep 2022 so it is very much in active development with a lot of people contributing https://github.com/popcorn-official/popcorn-desktop/graphs/c...
so while the original team might not be working on it, like a true free software, the code lives.
Encapsulates it all well I like this statement, total pottery
The blog post says they are worried about the ability to use the model to "use it for illegal purposes and hurting people". I think that they are referring to the ability to create all kinds of compromising pictures (porn) with celebrities, kids, etc. Am I misreading that? They don't mention copyright anywhere.
The conspiracy theorist would say that if you were doing something you shouldn’t, you wouldn’t mention it. Instead, you’d give a more palatable excuse to buy yourself some time while you figure out how to get away (legally) with the thing you shouldn’t be doing.
That's the only possible interpretation, really. AI models algorithmically remix input intellectual property en masse, without any significant amount of human creativity, the only thing copyright law protects. As such, the models themselves are wholly derived works, essentially a compressed and compact representation of the artistic features of the original works.
Legally, a AI model is equivalent to a huge tar.gz of copyrighted thumbnails: very limited fair use applies, only in some countries, and only in certain use contexts that generally don't harm the original author or out-compete them in the market place - the polar opposite of what AI models are.
It reminds me to follow the datahoarder maxim that if you don’t admin then servers, you don’t have the data. So now I clone stuff to a local drive.
https://texaslawreview.org/fair-learning/
Here is a short quote from an IP lawyer:
“In terms of the ingestion of publicly accessible code, Ochoa said, there may be software license violations but that's probably protected by fair use. While there hasn't been a lot of litigation about that, a number of scholars have taken that position and he said he's inclined to agree.”
https://www.theregister.com/2022/10/19/github_copilot_copyri...
Incorrect. In the absence of new legislation case law is how these things get worked out, but new statutes could be passed and could void prior case law.
https://www.copyright.gov/title17/title17.pdf
Let me expand on this a bit... if you read through the above text (feel free to search for the below terms) of the current laws around copyright you will notice that there is no discussion of:
https://en.wikipedia.org/wiki/Structure,_sequence_and_organi...
https://en.wikipedia.org/wiki/Abstraction-Filtration-Compari...
https://en.wikipedia.org/wiki/Idea–expression_distinction
These legal doctrines are the result of the details as hashed out in case law.
This is distinct from countries that use civil law jurisprudence. Common law jurisprudence relies heavily on case law.
Funny way to admit that your original statement was flat-out wrong.
There is no separate "civil law jurisprudence" and "common law jurisprudence". Common law by itself is by definition not jurisprudence. Civil law combines aspects of statutory and common law (as one form of precedent) into a single system. Some aspects of common law even make their way into criminal law. Instead of just Googling for buzzwords, learn what they mean before you try to bluster your way through an argument with them.
The law moves slow. Even were that to happen eventually, the laws will very likely be challenged in courts, and those will take a while to be resolved.
Finally, even if the US outlaws this, there'll be plenty of other countries where it'll be legal. There's plenty of infringement of US copyright in China, for instance. The same is likely to happen in regards to AI that's illegal in the US but legal elsewhere.
By the time the law catches up, model creation may become so easy to create by individuals instead of just by deep-pocketed corporations, it may be practically impossible to stop.
But sure, let's talk about outputs as well. From the second source we can see this from Tyler Ochoa:
"If there's only one good way to do it, OK, then that's probably not eligible for copyright. But chances are that there's just a lot of code in [the training data] that has used the same open source solution, and that the output is going to look very similar to that. And that's just copying."
I have seen some probable copyright violations from the output of Copilot, such as comments and some certain structural similarities that might be protected, although it is hard to say. But focus on the first part of what Mr. Ochoa is saying here, which is also laid out in this quote:
“In computer programs, concerns for efficiency may limit the possible ways to achieve a particular function, making a particular expression necessary to achieving the idea. In this case, the expression is not protected by copyright."
https://en.wikipedia.org/wiki/Abstraction-Filtration-Compari...
This allows for verbatim copies if they are utilitarian in nature!
As for why we should allow verbatim copies of utilitarian features... First, let's preface this with the substantial similarity of the structure, sequence and organization as established in Whelan v. Jaslow which amongst other things says that you cannot merely change the variable names if the expressive structure of the code remains the same. Now let's imagine 10,000 software developers who all implement Dijkstra's algorithm in C and then run it through clang-format. Aside from variable names, isn't it safe to assume that many of the implementations are going to be exactly the same?
As for why it was carefully selected... more often than not when I bring these things up people who feel upset about Copilot go off to cherry-pick some random quote out of context in order to support their upset feelings. Therefore I'm highlighting the important parts as to help people look beyond their upset feelings.
This is a complicated and nuanced matter. Attempting to channel everything through the lens of "this makes me personally feel bad and must be completely wrong" does not help the discourse. It may make you popular to a certain crowd but it might be unpopular to the public at large and it might also be incoherent from a legal standpoint, akin to bashing your head against a wall at a weekly meetup of the local heads-bashing-against-walls club.
There is plenty of room for discussion on what constitutes not only the legal interpretation of fair use and the idea/expression dichotomy but also the bigger picture. The knife always cuts both ways. Would it be acceptable to the open-source community if Microsoft could stop anyone from publishing Dijkstra's algorithm in C# because they wrote it first?
That's a very petulant way to defend cherry picking. I wasn't asking you to support one particular view; in fact that's the problem I was identifying. Your sources presented a balanced view, which you misrepresented by citing only the part that supported your own conclusion.
> focus on the first part
No, because the second part matters too. Here's Lemley and Casey again (emphasis mine):
<<<some purposes—say, ... a translation program that produces a translation of an entire copyrighted work—seem more substitutive than transformative, so that if they run afoul of the ever-broadening definition of similarity in music, fair use is unlikely to save them.>>>
Or the Register:
<<<"I actually think there's a decent chance there is a good copyright claim," said Tyler Ochoa ... the functional nature of the code means that reproducing it in a suggestion may not be seen as particularly transformative, which is one of the criteria for determining fair use>>>
Those are your own sources undermining - if not outright contradicting - your one-sided interpretation.
The limitation to market harms in the four-factor test for fair use should not be considered permanent. Law is, after all, a social construct. There's ample precedent for considering harms to the commons, to communities, and so on in other areas of law. Also, there might indeed be market harms. If a company open-sources some of their code but also hopes to profit by selling it in pre-packaged form or as a service, then AI-washed copying could constitute harm in even the most market-myopic terms. The "transformative" test is also pretty suspect in the context of AI-assisted copying, but this is getting long enough so I'll not go down that rabbit hole just yet.
> <verbosity about "utilitarian" copies which are not the issue here>
Enjoy your red herrings. I don't share your taste for them.
When Tyler Ochoa is saying that there is a decent chance of a copyright claim he is specifically talking about the output of the model.
Here is the full quote:
In the Texas Law Review in March, 2021, Mark Lemley, a Stanford law professor, and Bryan Casey, then a lecturer in law at Stanford, posed a question: "Will copyright law allow robots to learn?" They argue that, at least in the United States, it should.
"[Machine learning] systems should generally be able to use databases for training, whether or not the contents of that database are copyrighted," they wrote, adding that copyright law isn't the right tool to regulate abuses.
But when it comes to the output of these models – the code suggestions automatically made by the likes of Copilot – the potential for the copyright claim proposed by Butterick looks stronger.
"I actually think there's a decent chance there is a good copyright claim," said Tyler Ochoa, a professor in the law department at Santa Clara University in California, in a phone interview with The Register.
The use of the word "but" marks the transition from a discussion around the model itself to the outputs of the model.
Is it not also perfectly clear that Lemley and Casey are also of the opinion that the model itself is fair use?
Oh, you mean the very first thing I had to explain to you at the beginning of this exchange because you seemed to be ignoring it? Very little of these discussions has been about the models. Most of the discussion is about the outputs, and there the fair-use case is - as Lemley/Casey and Ochoa both concede - much weaker.
But by all means keep going on about feelings. We can all tell it's not others' feelings that are being hurt by mere disagreement.
IANAL but I would take any opinions on this right now with a huge grain of salt and treat them more as advocacy than actual predictions of any legal outcomes.
Whether there is a good case for it being considered fair use doesn't matter at all until its actually litigated and historically the result with fair use in relation to new technologies has always been a crapshoot.
The result could easily be affected by the actual cases that get litigated, and one well chosen lawsuit where machine learning software is shown to produce output that's too close to the material it was trained on could result in a completely different outcome.
Get 200 interested people backing up 1 TB each and you have your 200 TB backup.
With redundancy and error correction data added to the mix, you should be able to lose a certain percentage of participants and still have access to the full, error-free backup.
This should be next on my list since my current project depends on SD model and having data backed up gives me confidence that I can get rid of all their stuff if needed.
You don't even need to store it all at once on your computer: stream it and generate checksums on the fly. Then distribute the torrent, and seed sections at a time. It can also be distributed on IPFS.
I've seen a lot of torrents being used for distributing neural network (mostly stable diffusion forks).
Or multiple IPFS CIDs. I think you can have a "directory" (CID) that contains multiple CIDs, and only need the content hashes to build it.
You can also publish multiple CIDs and ask people to seed random ones; that's how Libgen does it (and is similar to the multiple torrents concept).
The same file can be used to seed both torrents and IPFS.
Just feels absurd to me because how is this different from any Human artist who you could equally say was "trained" on copyrighted material.
>Get it now when there are trustworthy sources. Once these kinds of things go underground, it gets much harder to get a trustworthy version.
People have already reverse engineered most text2image models and given enough hardware can train their own. There is no need for this hysterical take. As long as the internet exists you will be able to train these models.
Personally, I think commercial use cases should get license agreements from the authors for their training data, but I think non-commercial exemptions to advance the field of AI makes sense.
Irregardless of what I think though, the UK has set an international precedent, and the EU is apparently discussing about possibly extending it to commercial use cases as well. So there's that.
Where copyright may be applicable is when the models reproduce original art without modification that a reasonable person wouldn’t know the difference.
Something that would make any non-trivial model a legal nightmare.
Ugh. It feels like so many of these models are trying to censor NSFW material.
True, but that might not matter to the general public, legislators, or judges.
I just read a new article on the BBC related to this: "Deepfaked: 'They put my face on a porn video'"[1]
The person in question was not physically abused. Only a fake porn video with her face on it was released. But she was still emotionally distraught over it, and said: "You start thinking about your family," she says, holding back tears. "How would they feel if they saw this content?"
This is a real concern to a lot of people, and I doubt they'll be swayed by people pointing out that they weren't physically abused.
So, yeah, there'll likely be a massive backlash against some AI-generated content and the software that generates it, and laws will likely be made against it in some countries.
But in other countries and on the dark web it'll still exist. There's ultimately no way of stopping it, and eventually people will come to terms with its existence and widespread availability, no matter what the law of some countries says.
[1] - https://www.bbc.co.uk/news/uk-62821117
However, that will inevitably spread to include more and more companies and will also start happening outside the US. All the research around this is being published and there's a lot of open source code that facilitates this. So, it's just a matter of people optimizing and improving that and hardware getting cheaper.
I expect that once that market is big enough, you'll see cloud providers step up with provisioning infrastructure for this stuff. It will still be expensive to use but it won't have a lot of limitations.
AI driven porn is basically the obvious use-case where there are some big companies with lots of money operating in that space and plenty of incentive to make this happen. Morally that might actually be preferable to exploiting people as is their current way of operating. The likes of OpenAI won't be able to do much to stop that.
I read than when some guys from 4chan started running the leaked NovelAI model, they generated porn non-stop for 20 hs or more, no sleep, no eating.
Even without conspiracy theories, these models cost upto 10s of millions to generate, no suprise investors wouldn't like if you are giving it all for free, there should be some revenue model.
IMO it is like finding a computer in a world without them. It is mind-blowing and it will take over your mind if you let it. For some folks that results in lots of porn, for others it'll be fear. My guess is that it'll wear off eventually.
I don't see how NSFW photos can easily be stopped from being generated, with the model being open source. Maybe the model could be heavily pre-filtered to remove any photos that could possibly be used for NSFW images.
Which has a LOT of NSFW images in it. I suspect if you removed them from the training set it would go a long way to curb NSFW output but as you say people could easily train their own NSFW latent diffusion model.
They mentioned regulators here, and I would be curious to hear the story behind that.
Don’t want to go too tin foil hat, but it makes you wonder if a certain other AI company that claims to be “open” may be afraid of a company that actually is open and is applying political pressure.
Extremely likely that the FAANG lobbyists went into overdrive. The big guys know this will be an extremely important industry for the coming decades and don't want a new competitor swooping in with nothing to lose when established companies are forced to be cautious.
https://salsa.debian.org/deeplearning-team/ml-policy
Seriously, fire any coward lawyers erroring on the side of caution and get some that are versed in the NRA playbook.
That has to be a dig at OpenAI
Their new handlers can do anything to the contrary and are incentivized to curb release as well. The market is saying their new handlers are going to do that.
So we enjoy you proving us wrong!
I tend to thinking they are co-joined, but clarity helps.
I think the social harms side, they need to be careful to under-promise and over-deliver. The likelihood of preventing social harms is frankly close to zero, what they can do is make it more complicated.
Think like this: use stable diffusion to make one "actor" dance a lambada in the left field and save it. in a new state, make a different "actor" dance a lambada in the "right" field. Now using alpha masks combine the two actors. Can this represent sexy dancing? you bet your sweet bippy.
Promising not to release "two person sexy dancing" in this situation would be over-promising. Sure, it was done outside of the AI by masks. Will the law makers care?
(for actor and lambada and sexy dance, substitute whatever contextually means "harm" in a two-actor situation, semantically)