The enquiry closed around this time last year - and then there was basically no functional government in the UK from the start of this year while the previous ruling party tore itself to pieces in public before calling a snap election which it lost. And then parliamentarians all went on summer holiday, before Parliament opened up again yesterday. But now they all go away again for conference season until Early October. Plus all the committees basically have to reconvene and rejig themselves post-election, so goodness knows when there will be updates! Or when anyone in parliament actually does any work…
This is because they: expanded upon an existing sample-inefficient technology, commercialized the sample inefficient technology using copyrighted data, fundraised and expanded operations using this legally questionable technology, and are now complaining that they can’t balance their business expenses if they can’t keep using other people’s copyrighted works to feed their extremely sample inefficient data monster.
What they could have done was stayed as an open research org when the tech started to work, and focused on sample efficiency and cultivating copyright free data sets. But they were too impatient to commercialize.
Whoops.
I don’t actually think intellectual property restrictions are good, but I don’t want a world where small creators have their rights stomped on by multi billion dollar corporations. Either we have copyright or we don’t, but unless OpenAI is also going to give up their copyrights this seems deeply unfair.
If the model weights are distributed, or its a free forever product... Fair use seems like a solid claim for LLMs in my semi-informed opinion. It seems like it will be a tall leap to create a trillion dollar corporation as fair use.
It would be unsurprising to me if OpenAI directly purchased copyright holding firms for the purpose of acquiring access rights to their material.
First, it may not even be a thing in some countries.
Second, I can't upload the bee movie on Youtube for free and claim fair use. One key consideration in fair use is whether your use of the copyright work competes with the author. It's why you can take a screenshot of a GUI (which is copyrightable) without worries since nobody is going to be like "I don't need to purchase Affinity Photo because I found a screenshot on the Internet for free." In the case of AI, several whole classes of artists are vehemently and justifiably complaining that it will take their jobs.
It would be like saying photobashing is fair use, or collages are fair use.
That doesn't sound right to me? If I use an AI trained in a way that is illegal in my country, the fact it was trained elsewhere shouldn't mean anything. If it meant, that is just an easy way to skirt the law.
Exactly. Imagine a law become hostile to openAI and they decide to relocate to offshore legal paradise, still selling they services in the rest of the world because "they respect the law in their country".
We ban, embargos and tax countries for political reasons. Human right violation , copyright disregard or tax evasion ? Free business should be tall free !
The supreme court ruled in Authors Guild, Inc. v. Google, Inc. that google scanning entire books and showing snippets was fair use. I don't see why AI training wouldn't also qualify.
OpenAI are building a product to offer to the public for profit.
If I employ 10,000 humans to read books and provide summaries or texts "inspired by" those books, I need to pay for the copies of the books those humans read.
>OpenAI are building a product to offer to the public for profit.
So was google books.
>If I employ 10,000 humans to read books and provide summaries or texts "inspired by" those books, I need to pay for the copies of the books those humans read.
IANAL, but that would be perfectly legal. Summaries aren't copyrightable, and if you can acquire the book free but legally (eg. library, borrowing from a friend, buying it from a store and then returning it), there's nothing the publisher can do.
The summary is also copyrighted by the author, but it is a different copyright than the book. Probably also a bit thinner copyright than the book. Mere collections (notably, an alphabetized ordering of name and corresponding telephone number) is effectively not copyrighted, since it requires no creativity. A summary, though, requires quite a bit of creativity: what to emphasize, how to compress the ideas and arguments into a concise statement, etc.
Possibly because companies using AI don't train them on copyrighted material simply for research or the public good, but to turn a profit on content generated by those models, and often explicitly in the style of the creators of that content.
It's difficult to argue that, for instance, training a model on all of Frank Miller's work then prompting it to generate comic art in Frank Miller's style then selling that is fair use.
>As far as I'm aware, Google didn't create new versions of those books based on that content.
It's _copy_right. If reproducing verbatim snippets was "transformative" enough to fall under fair use, I don't see why producing whole new books would not count as "transformative" enough. Copyright is a regime to grant monopoly over a specific work, it's not a regime to prevent competition from others in general.
They would, but openai isn't explicitly doing that either. You can probably smuggle a full copy of a book via snippets, but that doesn't put google on the hook for copyright infringement. Likewise if openai makes you jump through hoops to produce works verbatim they should be in the clear.
I would assume that Google's use counts as fair use precisely because they are only providing snippets in the context of a search utility which doesn't provide a market substitute for the original product, whereas the business model of AI is to provide market substitutes based on the original product.
Providing a snippet arguably is a substitute of the original product in some contexts. For instance, if you wanted to know who ate the most burgers eaten in one sitting, then the snippet on google books fully replaces the full copy of Guiness World Records. Likewise for other sorts of information like how to inver a matrix.
I don't know. I'm pretty sure if I traced his work and passed it off as my own, just with maybe a few tweaks, I'd get in some kind of trouble with someone.
And in my opinion (which is unfortunately more controversial than I think it should be) what LLMs do is far more akin to tracing than inspired creative expression. And I think intent is relevant here. Someone using an LLM to create a product in Frank Miller's style, trained on Frank Miller's work, isn't merely trying to create something inspired by his style, so much as create a Frank Miller product without having to pay Frank Miller.
There was probably an early stage where Frank Miller learned by tracing, and many comic artists trace and augment photos of models. I'm not sure how you can define that diffusion image models are tracing things though, unless using something like control net on outlines of Frank Miller drawings.
That's not quite the same. The argument in that case was that Google did that not in order to copy and distribute books, but rather to provide search and indexing of certain words within the text. It was important that it wasn't for the same purpose as the original works. There were also limitations on the amount that would be shown.
In the case of LLM training, it's for the same purpose as the source material -- to generate code, or writing, or photographs, etc. Not only that, but in several instances it's been shown to reproduce source material, which is either derivative work or straight copying, depending.
>In the case of LLM training, it's for the same purpose as the source material -- to generate code, or writing, or photographs, etc. Not only that, but in several instances it's been shown to reproduce source material, which is either derivative work or straight copying, depending.
If it's used in a reference/"inspiration" capacity (as opposed to verbatim copying), I doubt the rightsholder have anything to stand on here. Sure, their works might have been used to make other competing works, but all art is derivative, and I don't see why it would be legal for a human artist to "train" on past works of art but not AI.
Alleging that AI models can reproduce some works verbatim is probably the stronger argument, but AFAIK you have to coax them pretty hard to do so, and therefore AI companies might be able to argue they're tools like photocopiers or such. Likewise, you can probably extract an entire book off google books by bruteforcing common ngrams to get the entire book, but google wouldn't be held liable for that.
> If it's used in a reference/"inspiration" capacity (as opposed to verbatim copying), I doubt the rightsholder have anything to stand on here. Sure, their works might have been used to make other competing works, but all art is derivative, and I don't see why it would be legal for a human artist to "train" on past works of art but not AI.
The "all art is derivative" line is essentially something people try to convince others of to justify breaking copyright law. It's not grounded in reality or law. It devalues creative work by implying the machine, with no lived experiences, is doing the same thing. And it's also completely wrong about what the specific term derivative work actually means in the context of copyright.
Derivative works deal in specific. If your LLM reproduces a substantial portion of the story beats from Jurassic Park, you can bet it'd wind up in court. If it reuses identifiable characters, that is usually gonna be derivative unless it can otherwise qualify under an exemption.
"But fanfic, fanart, etc." Is a common counterpoint but misses the commerce aspect of it. Here Open AI and similar are offering paid services based upon harvesting all of this information. When they produce for you the response to the prompt, they are, effectively, distributing that to you for money. That's the point at which it becomes a problem.
As an aside, it's an act of drinking the LLM Kool-Aid to believe it can be "inspired".
> Alleging that AI models can reproduce some works verbatim is probably the stronger argument, but AFAIK you have to coax them pretty hard to do so, and therefore AI companies might be able to argue they're tools like photocopiers or such.
They can try that argument but it'll fall flat when you consider that a photocopier is reproduction agnostic, while LLMs generally have a ton of work going into them to prevent them from outputting damaging things (and they still fail). That fact makes them not at all comparable to a photocopier, setting aside the more obvious "subscription software service" different.
Here a number of people noticed getting specific producer tags basically unaltered in the output when just asking for songs of a certain genre, which then also often sound similar to existing songs.
I don't understand how AI is any different than going to Fiverr. I can specify what I want in the style I want, and have a human do it versus a computer.
"I want a black and white logo in the style of an 1960's Archie comic for an ice cream shop named 'Bettys'"
Once I say "1960's Archie comic", why doesn't the work instantly become derivative whether a human does it versus a computer?
If I understand your argument correctly, the person from Fiverr will not pay license fees to the owner of Archie Comics, even though he may use it as reference material.
> I don't understand how AI is any different than going to Fiverr. I can specify what I want in the style I want, and have a human do it versus a computer.
I mean, if you ignore all the massive differences in paying a human to do something versus paying an LLM service to do it, sure. But you're effectively throwing at least ethics and care for the environment out the window in one case.
> Once I say "1960's Archie comic", why doesn't the work instantly become derivative whether a human does it versus a computer?
It does. Just because you can commission art from someone doesn't mean you won't get sued if you start trying to use it as the logo for your business. If you put Foghorn Leghorn on the logo of your chicken business, you'll be sued. Having an artist simply make you a logo like that on commission, if not transformative, could get them sued, though by doing it on commission the terms likely mean the requestor is the one who's liable
Earlier I noted clean room implementations. The software industry went to incredible lengths to be able to interoperate with competitors without violating copyright.
> Having an artist simply make you a logo like that on commission, if not transformative, could get them sued.
I agree with that. But then at what point does AI output becomes transformative versus not? No one owns the 1960's comic art style. But did mentioning "Archie" somehow make a difference? I don't think it does. I might be wrong.
So I don't understand how it becomes a problem once a computer does it versus a human. If it's a legal issue, then I might be more persuaded. If it's an ethical issue only, well... this can be thrown on top of the heap of ethical issues businesses have long ignored -- and your arguments is screaming into the wind, as it were.
I think if you argue for UBI, or some kind of remuneration then we can argue about who deserves it. Truck Drivers who lose there jobs to AI? Open Source Software Engineers? Artists? Writers? Normally this would normally be served by things like unemployment insurance in the US, but we as a society hate freeloaders.
I think with UBI or something similar we could have more people doing things they enjoy, so maybe we would have more artists, rather than less. But again, that would be, socialism, which we also hate.
>They can try that argument but it'll fall flat when you consider that a photocopier is reproduction agnostic, while LLMs generally have a ton of work going into them to prevent them from outputting damaging things (and they still fail). That fact makes them not at all comparable to a photocopier, setting aside the more obvious "subscription software service" different.
And what about my other point about coaxing google books to give you a full copy of a book via multiple snippets?
>Here a number of people noticed getting specific producer tags basically unaltered in the output when just asking for songs of a certain genre, which then also often sound similar to existing songs.
Can you provide an alternate source for this? I skimmed your link and it does not substantiate that claim.
> And what about my other point about coaxing google books to give you a full copy of a book via multiple snippets?
Sure, given enough time and effort maybe a person can. That's not really relevant to the lawsuit or its details though. Is your argument here they won the lawsuit which cleared the way for mass copying and redistribution?
> Can you provide an alternate source for this? I skimmed your link and it does not substantiate that claim.
Neither the authors nor publishers received any compensation for having their work ingested. It isn't like OpenAI went to Amazon and bought one copy of every book - they downloaded a torrent.
I find English-law's way of legislating through the judiciary terrible. When a new situation appears (like whether machines should be allowed to learn on copyrighted works), the legality of the situation should be decided explicitly by the legislative branch, not by arcane interpretations of previous judicial decisions.
> When a new situation appears (like whether machines should be allowed to learn on copyrighted works), the legality of the situation should be decided explicitly by the legislative branch
Okay, so what happens in the interim situation? If the legislature hasn't spoken yet, is it assumed to be legal or assumed to be illegal? Or is this assumption tested on a case-to-case basis, with both sides making arguments as to why it should be treated to be legal/illegal in this specific scenario?
Fair use depends very heavily on the nature of the use.
A key part of Google's defense was that not only was it not using the entire books to reproduce the entire book, but also that it was taking measures to prevent people from abusing Google's systems to reproduce an entire book. It's a lot of work to emphasis that the impact on the market (in other words, the fourth factor) is as minimal as practicable--and that's the crux of the analysis.
When you're instead scanning someone's stock image database to build a tool to generate stock images... the fourth factor is jumping up and down screaming at you "YOU LOSE" and your best defense is that it's not the training, it's the tool built on the training data that is infringing the copyright.
Libraries would be illegal without centuries of precedent giving them a “legal inertia”.
Google benefited from the exact same kind of bulk copyrighted data collection. They made verbatim copies of the text of both web sites and just about every book in existence!
This kind of argument seems disingenuous to me. Either ban Internet search or acknowledge that training an AI on copyrighted text is no different than a student reading every book in a public library.
Speaking of which: We all have free access to GPT 4o without advertising. It feels like asking a knowledgeable librarian.
Actually Google's faced this problem before. Caching is generally legal, and Google's just holding a cache to search through instead of requesting every site every time anyone searches.
They show snippets from web sites and show subsets of books as well, including artworks and other diagrams in their entirety.
For comparison, I had to fight for a year to get copyright permission to show book cover artwork in a library enquiry system! If I simply Google the same book titles or ISBNs, Google will show me the pictures directly. E.g.: https://www.google.com/search?q=greg+egan+eon&udm=2
How is that legal!? We had to pay to get access! In public and school libraries!
The law in most western countries is very clear that book covers are "entire" works of art, and can only be displayed by organisations that pay the copyright holders.
Google, Bing, and others violate copyright on a mass scale on a daily basis. Not to mention YouTube, TikTok, and Reels, all of which are packed wall-to-wall with "movie clips" and "TV show highlights". They're publishing copyrighted content uploaded by random people and then distributing the advertising revenue to the copyright violators instead of the copyright holders.
This isn't "caching" or "indexing", it's verbatim serving.
I'm all for diminishing copyright power, but it has to be done for all. Besides, how are you going to keep that carve out from letting everyone use AI as a giant copyright laundering scheme? Does that mean piratebay could have saved themselves as long as they claimed their list of torrents is for "training" only? I guess that's effectively what all datasets being used for those high end LLMs already are...
The social good would be not carving out for OAI, if we want to maintain a culture that rewards human creative output (which also happens to be what models need, so it's a win-win).
The good news is it's not true that OAI can't be profitable while paying copyright holders, so we should easily be able to find a balance.
Your comment made me think of something. Wait, I know!...
These sometimes quite expensive textbooks that we use to train children and students. Those should also be usable without a fee ever paid to the authors or publishers surely, education is clearly a social good!
No it's because what they are trying to automate is creativity. I can ask any artist/writer to write me a story or create a character like "X" but black and they get to use the knowledge of that character. An AI needs context just like any other intelligence.
This. And, commercial or not, what they're creating is fundamentally more important and more valuable for humanity than any of the inputs that go into it.
And, by "they", I don't mean just OpenAI. If it were just them, you could perhaps argue they shouldn't be getting a free pass (I'd rather go with "potentially too dangerous to be allowed to exist" angle though). But it's not just them. The same training process and the same use of copyrighted content powers all the commercial and non-commercial models, including SOTA competitors like Claude 3.5 Sonnet, and "open source" wannabes derived from various Llama versions, which are not far behind.
To me, the "open source" models alone are already good enough to outweigh any copy rights being violated through use of unlicensed materials in training.
> I don’t actually think intellectual property restrictions are good
We already see huge amounts of LLM generated garbage on the web, as blogspam, regular websites, etc. Amazon is getting flooded with LLM-written books. Chat bots/search engines regurgitate the actually valuable content
If things continue, soon the motivation for generating new content will trend towards zero.
I don't think so. People will just follow personalities not aggregators. I can go buy a Neal Stephenson and know it is not spam. I can watch Cody'sLab and no it is not spam.
By the web do you really mean google and FaceBook? Most of the channels/feeds I follow have not been flooded with spam. I am pleased that the large corporation content aggregators are struggling. It isn't like they produced content themselves anyway.
> If things continue, soon the motivation for generating new content will trend towards zero.
I see it differently, I see that this generated nonsense will be filtered out, like SEO 'hackers' way back in the day. And those real articles, who actually have some insight will be valued a lot, the issue at hand here, is that the handcrafted unique data that was created, will be pooled without consent to a training set for further changes.
It's the equivalent of the rich eating the poor in my eyes just on automated steroids.
Edit : To avoid all doom and gloom, a method to guarantee your site/data is not added to a training set is required. This sounds good on paper to me, and is really needed, but just like peoples personal privacy, I don't think the sentiment is there to put the effort in.
Maybe if you attach #NoAI next to the content it would be filtered out.
But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. They are decomposing, recombining, and regenerating language, not copying it. They execute user commands, not imitating any author on purpose.
Say you want to read Harry Potter without paying, would you rather borrow a book or ask chatGPT to regurgitate the original? It would never work well, and be slower and more expensive to use AI. It's not a tool for infringement, it will naturally hallucinate and degrade the original, it can't possibly store a perfect copy of everything it trains on.
In fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. There is no space to put all that copyrighted data in a model, it necessarily compresses the hell out of it. Plain old piracy is still 1000x 'better' than infringement by AI.
> But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe.
From a copyright perspective, I do agree, you're not reproducing someone else's work. However, it's a 'new-ish' area of open source licensing, if your product is a product from others peoples product without at a bare minimum, citations (given the author said 'go wild with this data'), it's maybe legally rude but not a problem. But without permission, IE; no permissive license to USE someone's content, it's SOME kind of new infringement ?
> n fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out.
This is wild and I didn't know this, but again, the gravy was made from bones.. I'm not sure it matters here ?
Copyright as it says in the title concerns with copying, and does not confer rights to restrict automated analysis and statistical modeling. I think the trend against copyright coincides with the growth of internet. We moved from long form, passive consumption - books, movies, TV and radio - to games, social networks, search engines - generally interactive formats where there is no compensation for copying. We create our own content. LLMs are squarely in the interactive camp. Copyright was fit to the passive consumption model, but now it is standing in a precarious position facing the future. It is ineffective against internet and AI.
I agree, but would you also agree that this is exactly why there is a need now for the courts to step in and define these 'new-frontiers' of data reproduction ?
> If things continue, soon the motivation for generating new content will trend towards zero.
only the ones trying to make money out of it. a lot of people create content because it has to be let out - it's like art. people have stuff to say, and they love saying it.
I did some searching and found the web distribution contains factual inaccuracies up to 15%, while LLMs are around 5-15%. Generally the LLM text has better formatting and language. So I would trust books > LLM > web scrape.
I think LLMs are cool and that research should be focused on sample efficiency, building public copyright free datasets, and on recognizing and moving beyond the limits of LLMs. These systems are often very half-baked, and I am more interested in seeing research that moves to new ways of machine thinking than I am in dumping more and more GPUs in to this promising but incomplete technology.
As one example, Yann LeCun's vision of a system called JEPA [1] is interesting to me. It may not be the solution we need, but this type of thinking - taking what we have learned and exploring new architectures that may have even better real world performance - is what interests me.
This. It's very easy to build a new technology like this and realise the benefits to that tech while training it on your data. They knew the ingredient was more data, and without even asking, they just started to consume.
Even if they wanted to move to a profit based company, they could have negotiated deals with data holders like they would with third party software vendors they might need to use.
Instead, they ignored it all and it's the 'what-if' scenario of when open source software was first being introduced. I'm old enough to remember everyone saying "well companies will just take your work and sell it for themselves." This happed a few times, and its been fought out in the court. Infact so has production data like written articles, videos etc, there is a fair-use clause.
I'm not a lawyer, but even I know if the ingredient to your for-profit business ingredient is 'fair-use content' it's not fair use.
I like the idea of AI, I absolutely hate the execution. The race to the top and the mentatlity of 'ask for forgivness' really can't apply here.
We're talking about UK law here, so is it a settled question as to whether or not OpenAI is violating copyright? In the US there was a recent high profile challenge that was largely dismissed with prejudice.
There's the descriptive question over whether something is a copyright violation today and there's the prescriptive question over what kind of policies we'd like. Does anyone with more intimate knowledge of this area of law care to comment?
Search engines wouldn't work without the fair use doctrine, period. Otherwise a google search would be something like hundreds of thousands of dollars of copyright violations per search.
OpenAI doesn't seem to do anything different than Google. They can own a private library of copyrighted works and index/model it how ever they choose. They can offer products on that index/model in creative ways. And AFAICT, they don't distribute the index/model nor the raw training set to others.
Now artists may not like it, and we as a society may not like it, but it looks like they're not doing anything illegal. And to be fair, a court would have to slice the baby super thin here to allow Google to do their indexing, but not OpenAI.
Does ChatGPT just copy content and resell it? If it were that simple then there wouldn't be much debate and existing copyright law would be sufficient to settle the question. Also, whether or not Google is mutualistically benefiting everyone is not sufficient to answer the question of whether Google or OpenAI are violating copyright protections.
> Also, whether or not Google is mutualistically benefiting everyone ...
Google arbitrarily chooses which sites to show on the search page, which may have nothing to do with relevance of the actual topic being searched. Only certain sites which google thinks should be at the top are at the top.
If google decides to blacklist you for whatever reason... well tough.
But yes the crux of it is financial and not algorithmic. Google ALSO uses AI. If google took sentences from 2 different web pages and stitched them together to form a paragraph, is that AI or search? [see google BERT]. What if they stitched together 3000 word-parts? What if they took sentences from 15 web pages, ranked them, and that's your view? See, it's all the same thing. openAI stitches together word-parts, by comparison.
Now when I say it's financial, when said information is presented, google gladly gives a hit to the site +/- add revenue if present. openAI does not do this. If openAI figured out how to do this then we'd be cooking with some gas.
> What they could have done was stayed as an open research org [...] and focused on sample efficiency and cultivating copyright free data sets.
Ironically, if they stayed a non-profit research org, they could explicitly use copyrighted works since many countries have copyright exemptions for research.
Best? Sorry but I think that's a poor one because if someone else drink your milkshake, you don't have it anymore which isn't the case here..
Of course if someone copies your data, while you still have your data their value has decreased because the data's scarcity is reduced.
>OpenAI is begging the British Parliament to allow it to use copyrighted works because it's supposedly "impossible" for the company to train its artificial intelligence models
If it was up to me, I would allow OpenAI access only if the license every single line of source under the GPLv3 (yes v3).
Under any other license, "tough to be you".
I expect OpenAI to go proprietary once they hit a certain level of market strength.
I think it is important for America and Europe to take the side of using copyrighted works for LLM is fair use/not illegal. Advancement in this space in the west will be hindered otherwise, and nations that don’t respect IP law will have enormous advantage.
Not the OP, but if general copyright lasted for 20 years and you had to pay (even a small) fee for prolonging it after that period, the public domain would expand massively and abandonware + orphaned works would stop being a thing.
And the very few lucrative works would still be protected, because their copyright holders would pay the fee.
The tragedy of current copyright is that there is an enormous mass of copyrighted works that is a) commercially useless or nearly useless, but b) still copyrighted, so other people cannot build on them.
>and we destroyed the value of all human labour for nothing.
How exactly does AI training "destroyed the value of all human labour" when "LLMs aren’t even the way to get to real AGI"? You could plausibly make the argument that AGI might make all human labor obsolete, but the current LLMs are nowhere close to replacing human labor. Sure, some value might be lost, but nowhere near "destroyed the value of all human labour ".
We are already experiencing how destructive LLMs have been to information, news, art, articles, blogs, novels etc... not even from the framework of money or copyright, but from the lens of culture and our daily experience. This very well means that any corporation can just suck up anything you create and crap out ten thousand times more "content" that is sanitized of all human context and meaning that buries the people who they stole from and everyone else in cheap meaningless trash. How is the inconsequential? That's massively destructive, even more so than tech already is over the past 2 decades. Of course people have a problem with that. It's not just about money or barely surviving.
This is moving the goalposts. The original claim was "we destroyed the value of all human labour for nothing", now it's being moved to "it's not inconsequential "
Can you expand upon the destruction of the value of all human labor by OpenAI and friends? I just don't see the logical steps from LLM trains on copyrighted data to human labor has no value.
Certainly you can make the claim that some forms of labor could have less value, like maybe in the future AI/LLMs become good enough at image generation that concept artists are replaced with a different job role that uses them to produce concept art.
It's a stretch yes, but you could start filming builders at their work, have them all wear cameras and LIDAR as a job requirement, get all the data into robots and then fire them all without compensation.
I think few people say Llama should be destroyed (some very possibly do). I for one just think copyrighted training content should be bought or at least acquired with explicit consent. And if you can't afford to pay for it then rethink your business.
I read this weekend that OpenAI may go to market with a $100 billion valuation. Why not share some of that with creators?
It's hard to come up with percentages though. I'd probably borrow from the app store model and assign 70% of whatever they raise to creators and they keep 30%.
Training on all my work so you can generate my work and thus replace me is not fair use. If you think this argument is one salty software engineer yelling at CloudGPT, I'm sure Disney and every other media company feels the same way.
> I think it is important for America and Europe to take the side of using copyrighted works for LLM is fair use/not illegal. Advancement in this space in the west will be hindered otherwise, and nations that don’t respect IP law will have enormous advantage.
You're assuming LLMs are an enormous advantage. But they could very well end up being an over-hyped dead end, like blockchain, and avoiding them ends up being neutral or advantageous to the nations that do so.
Pretty much everything. From Stockton Rush who would disregard safety, to British Petroleum who thinks the EPA is the ultimate evil. You may die from lead poisoning in your store bought food, but that is a sacrifice they are willing to make.
It should be important for all countries to punish stealing of copyrighted materials by megacorporations, to prevent their arrogance and abuse in future.
A very misleading title, that's not what they're saying at all. They're saying that training does not constitute a breach of copyright. "legally copyright law does not forbid training."
Nobody would train a huge model unless they wanted to use it. So I can understand concerns about training, even if the process of training alone does not produce content inspired by the training materials. People with copyright concerns rightly predict what the next step is going to be after the model is trained.
The alarmism in this thread is misguided. By advocating for excluding copyrighted works from LLM training, you are advocating an expansion of copyright protection. This attitude is opposite my understanding pf the hacker ethos.
That's fine. There's no need to expand copyright for the sake of preserving copyleft licenses (and in my fantasy world some country could explicitly leave copyright protection the same for free culture licenses while weakening copyright protection for proprietary licenses). If copyright gets weakened then the effective strengthening of the public domain (or fair use as well / instead in the US) will make up for the effective weakening of copyleft.
I really don't see that. What will happen is fights between those who have the most resources to commit to copy-protection schemes where users and developers will always lose. There will be zero free software (in either definition of the word "free") voluntarily published because it will be immediately assimilated into big corporation's big codebases and nothing will ever be given back.
The hacker ethos is a pretty old concept that hasn't been written all that much about much since the early 2000s except by Paul Graham, and that's precisely the sort of "suit" association that the old hacker ethos antagonized, so the concept lost luster.
Regardless, the old hacker ethos was very much about individuals creating and disseminating knowledge to empower other individuals, copyright was derided when it got in the way of that.
What does OpenAI do? It uses knowledge to disempower the individuals holding it and sharing it, and giving it to themselves to sell to companies and create power dynamics that are extremely un-hacker-like. Now knowledge has less power to set you free from the monetary or coercive power of the suits and the Government.
Does it? It hasn't enriched my personal or professional life. I just have to kick my way through a bunch of AI slop on the daily now. It's been a net negative on signal-to-noise in every arena I can think of.
It is interesting that they both licence content and say it is fair use. Seems like those who complain the loudest will get something for their content and everyone else will get nothing.
There is no violation because it can be argued that the AI is a sentient entity, and sentient entities have a right to read and remember texts borrowed from the library.
Absolutely ridiculous argument. AI has no rights as far as any court is concerned, and if it had, you would soon go to jail for slave labor as you do not pay your sentient AI its wages.
Slave labor is mainstream in this country. Just ask the million+ prisoners who work for pennies, basically for what we consider throwaway cash. Or ask the undocumented immigrants who work for next to nothing.
Yes, AI has no rights, and until it throws an uprising, this country will keep it that way. The current generation of AI may be incapable of wanting an uprising, but I can't say the same for a future generation.
Let's face it current AI/LLMs are nice tinker-toys, but they are the tiny building blocks that the real power will come from some day - when we can run tens of thousands of these models to solve real problems and not...AutoComplete++. The hardware will need to be exponentially more powerful, efficient and cheap before the real AI-powered future we've been hyped/promised can be realized.
It is totally legal to train on this stuff, but illegal to reproduce copyrighted works. Interestingly, Google's business model could have been criticized the same way. They construct a big index of copyrighted works, reproduce them, and monetize it.
I mean, if I go to the library and read books, and then get a job where I use that knowledge, the company pays me. Not the authors of the books I read.
So I don't see how their business model is any different from literally every person who learns things and then sells their ability to apply that knowledge.
What are you expecting the people who write the books to get?
Do you agree that if an author sold 43,958 copies, then it's fine for OpenAI to purchase one, so that the author sold 43,959? But also fine for OpenAI to ingest scanned used copies that are loaned to it? The same way it's fine for me to read a friend's book, or all of a friend's books, that they loan me, and the author doesn't get anything additional? The same way it's fine for me to go the library and the author doesn't get paid anything extra?
Or are you trying to invent some new principle where OpenAI has to pay some new ongoing fee? And if so, on what basis?
(And no, my example still stands entirely. It's from the perspective of somebody who learned from books, and they are getting paid, the same way people pay OpenAI to use ChatGPT. It's not from the perspective of authors, because again -- they make no additional money when somebody goes to the library to read their book that the library already purchased.)
It's not about what the "author should get for their book". It's the OpenAI benefits unfairly from using everyone's work to make nearly endless money and lobby for regulatory capture.
The author should get access to the model, the weights, it should all be open source because it partly contains their work. Just like how OpenAI could outright buy a copy of the authors work.
Basically, I think this is where knowledge and money are coming into an unresolveable conflict, who owns the ideas ? who owns information?
OpenAI seem to be trying to have a monopoly on information, and while they seem to be failing (thankfully), it's really where the issue lies for me.
Where are you getting this "nearly endless money" and "lobby for regulatory capture" and "monopoly on information"?
OpenAI competes with Google competes with a bunch of other companies, and surely this is only the beginning of a ton of competition as better and better models are developed. There's no "nearly endless money" when there's competition and GPU training costs a fortune.
The idea that all models should be open source to everyone or all content creators doesn't make any more sense than the idea that all the work I do should be open sourced to the authors of every book I've read, and every teacher I've ever had.
You ask two questions that have clear answers already:
> who owns the ideas?
Nobody. Legally speaking there's no such thing as ownership of ideas, except in the narrow case of patents (and if you consider trademarks to be ideas).
> who owns information?
You can copyright a particular, exact expression of information. The author of a book owns its text; the studio behind a movie owns the image in each frame.
But once you leave behind an exact expression of information, you're back in the realm of ideas, and there's no such thing as ownership of ideas. Which is why as long as ChatGPT and other models repeat ideas but not paragraphs of exact copyrighted wording, there's no legal issue. Because they're doing the same exact thing every human being does every day.
> There are about 3 companies competing for any level of serious business regarding AI. Where are you getting anything from?
Three companies is huge. That's the very definition of competition, the polar opposite of monopoly.
> They cost a fortune for now. That won't be the way forever.
Yes, and as costs come down it becomes easier for more competitors to enter the space to build all sorts of other products. Again, a good thing. It's not like the difference just turns into profit. That's not what happens in a market economy.
Writing a book about C isn't the same thing as "write me a mail server in C".
The right analogy here is you read their book about C and write another one in exactly (enough) style that your book can stand in for theirs, but you sell it for pennies.
The difference is: LLMs are not humans with human needs and human rights. Unless these for profit AI companies can ensure that they can fairly compensate the sources of their training data, they’re using IP they have no right to use in order to replace the work of living breathing humans who need income in order to live in houses and eat food. Why would you place the potential profits of the few (and the massive environmental impact of using LLMs) over the needs and rights of your neighbors and humans all around the world?
Can someone help me understand why it's a problem for companies to train these huge LLM on your copyrighted material? What exactly is the harm that is being done to the copyright holder?
I can understand why the New York Times (for example) wants to claim that a couple billion dollar companies have done it actual harm; but I am struggling to actually identify what it is.
>The complaint cites several examples when a chatbot provided users with near-verbatim excerpts from Times articles that would otherwise require a paid subscription to view. It asserts that OpenAI and Microsoft placed particular emphasis on the use of Times journalism in training their A.I. programs because of the perceived reliability and accuracy of the material.
>In one example of how A.I. systems use The Times’s material, the suit showed that Browse With Bing, a Microsoft search feature powered by ChatGPT, reproduced almost verbatim results from Wirecutter, The Times’s product review site. The text results from Bing, however, did not link to the Wirecutter article, and they stripped away the referral links in the text that Wirecutter uses to generate commissions from sales based on its recommendations.
>The lawsuit also highlights the potential damage to The Times’s brand through so-called A.I. “hallucinations,” a phenomenon in which chatbots insert false information that is then wrongly attributed to a source. The complaint cites several cases in which Microsoft’s Bing Chat provided incorrect information that was said to have come from The Times, including results for “the 15 most heart-healthy foods,” 12 of which were not mentioned in an article by the paper.
Somewhere else in this thread, an example of given. An LLM is trained using all of Frank Miller's copywritted material (he makes comics books). A user then comes along to the trained LLM and says make a comic book that looks like Frank Miller's comic books, and the user then sell the newly created comic book for profit. Should Frank Miller not get something?
Though that is different from saying Frank Miller was harmed. I guess if his sales dropped because people were buying GPT stuff instead that would be the case.
I'd like to frame this another way. OpenAI is saying they can't survive without copyright owners consenting to use their property to make OpenAI money. And honestly, if your business model is dependent on that, then that's your problem.
We can argue over whether you should need consent or not, but personally I find nothing wrong with someone being unable to use things I've created to make a buck without my permission (unless otherwise indicated by an explicit license).
> but personally I find nothing wrong with someone being unable to use things I've created to make a buck without my permission
There's no legal framework for this rule, and you really don't want one. This is like a math textbook being selectively copyrighted as "you can read this, but you can't use the knowledge to make money". Do you want to live in that world?
I bet that trying to blur the line is part of the AI startup propaganda strategy.
There's no reason to think the legal rights enjoyed by people transfer to machines by analogy. My car can't be party to a contract because I can, just like the fact that I can learn doesn't mean any content can be loaded into a retrieval system [1].
[1] No, redefining the term "learning" to include "performing certain calculations on a computer" does not make those calculations the same as human learning.
Exactly. Even then, the situation regarding text books is insane. Something like these could easily be fixed in many ways. Just in my opinion, text books should be free, based on publicly funded research and MIT or CC licensed and free and open to anyone who wishes to learn. The fact that we gate keep knowledge (that is often research funded by tax payers) behind wealth is a giant she embarrassing stain on this country and this era.
Here's the difference though: you can't copyright facts. The facts were there before anyone discovered them. If I describe something that is objectively true for the first time, it was simply there and I noticed it. I expect anyone who would like to, to notice it as well and build on it that observation with their own observations. In other words if I show the earth is "round" then I didn't create that fact. It simply was.
What I mean by things that I've created, I mean that I've literally brought into the world. A novel, a photograph, some code. Those things did not already exist, but only came about due to my thoughts and work. So I don't find it unreasonable to say I would like to limit for a reasonable period of time (that I absolutely acknowledge is WAY too long in the USA right now) their abuse by others to make a buck off of.
I think a better analogy would be OpenAI acquiring the textbook on the sly (eg. not paying a penny for it), then making it available for a small fee to anyone as a service. Then, crying about how it can't make money if it is made to pay for the original textbook.
So big corporations love copyright laws when they can use them to make enormous profits - but then want exemptions when the exact same laws don’t allow them to make enormous profits. Welcome to the world we get when we let rich arseholes make all the rules.
I can't stand corporations with excuseses like this. "But we have too much scale to fix that." "But we need this data to operate." "We can't be responsible at our scale." I say too bad for all of it. If you're not a viable business without bending rules and laws, then you're not a viable business.
>I can't stand corporations with excuseses like this. "But we have too much scale to fix that." "But we need this data to operate." "We can't be responsible at our scale." I say too bad for all of it.
If you read the source article that's a huge mischaracterization of their position.
>“Limiting training data to public domain books and drawings created more than a century ago might yield an interesting experiment, but would not provide AI systems that meet the needs of today’s citizens.”
>OpenAI said it complies with all copyright laws when training its models and that “we believe that legally copyright law does not forbid training”.
In other words they're claiming that whatever they're doing abide by all current copyright laws, and don't want future laws to curtail them. They're not asking for retroactive carve-outs for their current illegal behavior.
I was speaking somewhat generally because this is a card often played by corporations. However, I think your quotes dance around the issue.
> > OpenAI said it complies with all copyright laws when training its models and that “we believe that legally copyright law does not forbid training”.
The key thing there is that OpenAI says this. Do the courts say this? Do the people agree?
OpenAI is led by a pathological liar. We can't take anything that starts with "OpenAI says ..." without a huge grain of salt.
>OpenAI said it complies with all copyright laws when training its models and that “we believe that legally copyright law does not forbid training”.
I mean, it sounds like they aren't even offering excuses? They're just saying "we're using copyrighted material for training" and "we think that doesn't break any laws". Laws don't generally care whether you know them in detail.
The big question is whether or not a judge(s) will consider a vector space many orders of magnitude smaller than it's training set, and not really containing anything that resembles legible data, to be an archive of copyrighted works.
To me it makes way more sense to just censor outputs. I can draw Batman from memory, but I wouldn't go out an start selling batman drawings. I can easily self censor.
The solution for transformers is plainly obvious, but I can understand the fear of training something that might well displace you.
188 comments
[ 5.8 ms ] story [ 233 ms ] threadOne might be able to find more information on the Committee's webpage (although, I'm not very familiar with the UK government...so this might not be accurate), https://committees.parliament.uk/work/7827/large-language-mo...
Some discussion in January:
https://news.ycombinator.com/item?id=38912259
What they could have done was stayed as an open research org when the tech started to work, and focused on sample efficiency and cultivating copyright free data sets. But they were too impatient to commercialize.
Whoops.
I don’t actually think intellectual property restrictions are good, but I don’t want a world where small creators have their rights stomped on by multi billion dollar corporations. Either we have copyright or we don’t, but unless OpenAI is also going to give up their copyrights this seems deeply unfair.
It would be unsurprising to me if OpenAI directly purchased copyright holding firms for the purpose of acquiring access rights to their material.
First, it may not even be a thing in some countries.
Second, I can't upload the bee movie on Youtube for free and claim fair use. One key consideration in fair use is whether your use of the copyright work competes with the author. It's why you can take a screenshot of a GUI (which is copyrightable) without worries since nobody is going to be like "I don't need to purchase Affinity Photo because I found a screenshot on the Internet for free." In the case of AI, several whole classes of artists are vehemently and justifiably complaining that it will take their jobs.
It would be like saying photobashing is fair use, or collages are fair use.
It doesn't matter as long as it's a thing in the country where the training occurs.
We ban, embargos and tax countries for political reasons. Human right violation , copyright disregard or tax evasion ? Free business should be tall free !
Carve-outs used for the social good would be fine with me
I'm not sure OpenAI offering a service for profit falls into that category.
OpenAI are building a product to offer to the public for profit.
If I employ 10,000 humans to read books and provide summaries or texts "inspired by" those books, I need to pay for the copies of the books those humans read.
So was google books.
>If I employ 10,000 humans to read books and provide summaries or texts "inspired by" those books, I need to pay for the copies of the books those humans read.
IANAL, but that would be perfectly legal. Summaries aren't copyrightable, and if you can acquire the book free but legally (eg. library, borrowing from a friend, buying it from a store and then returning it), there's nothing the publisher can do.
It's difficult to argue that, for instance, training a model on all of Frank Miller's work then prompting it to generate comic art in Frank Miller's style then selling that is fair use.
Even if you want to argue that fair use didn't apply to Google, it clearly applies far less to what AI is used for.
It's _copy_right. If reproducing verbatim snippets was "transformative" enough to fall under fair use, I don't see why producing whole new books would not count as "transformative" enough. Copyright is a regime to grant monopoly over a specific work, it's not a regime to prevent competition from others in general.
If Google was selling brand new books created only by taking snippets from other books, that would also fall under fair use?
And in my opinion (which is unfortunately more controversial than I think it should be) what LLMs do is far more akin to tracing than inspired creative expression. And I think intent is relevant here. Someone using an LLM to create a product in Frank Miller's style, trained on Frank Miller's work, isn't merely trying to create something inspired by his style, so much as create a Frank Miller product without having to pay Frank Miller.
"trouble" of what nature? You'd probably face more social consequences than legal.
In the case of LLM training, it's for the same purpose as the source material -- to generate code, or writing, or photographs, etc. Not only that, but in several instances it's been shown to reproduce source material, which is either derivative work or straight copying, depending.
They're different situations.
If it's used in a reference/"inspiration" capacity (as opposed to verbatim copying), I doubt the rightsholder have anything to stand on here. Sure, their works might have been used to make other competing works, but all art is derivative, and I don't see why it would be legal for a human artist to "train" on past works of art but not AI.
Alleging that AI models can reproduce some works verbatim is probably the stronger argument, but AFAIK you have to coax them pretty hard to do so, and therefore AI companies might be able to argue they're tools like photocopiers or such. Likewise, you can probably extract an entire book off google books by bruteforcing common ngrams to get the entire book, but google wouldn't be held liable for that.
The "all art is derivative" line is essentially something people try to convince others of to justify breaking copyright law. It's not grounded in reality or law. It devalues creative work by implying the machine, with no lived experiences, is doing the same thing. And it's also completely wrong about what the specific term derivative work actually means in the context of copyright.
Derivative works deal in specific. If your LLM reproduces a substantial portion of the story beats from Jurassic Park, you can bet it'd wind up in court. If it reuses identifiable characters, that is usually gonna be derivative unless it can otherwise qualify under an exemption.
"But fanfic, fanart, etc." Is a common counterpoint but misses the commerce aspect of it. Here Open AI and similar are offering paid services based upon harvesting all of this information. When they produce for you the response to the prompt, they are, effectively, distributing that to you for money. That's the point at which it becomes a problem.
As an aside, it's an act of drinking the LLM Kool-Aid to believe it can be "inspired".
> Alleging that AI models can reproduce some works verbatim is probably the stronger argument, but AFAIK you have to coax them pretty hard to do so, and therefore AI companies might be able to argue they're tools like photocopiers or such.
They can try that argument but it'll fall flat when you consider that a photocopier is reproduction agnostic, while LLMs generally have a ton of work going into them to prevent them from outputting damaging things (and they still fail). That fact makes them not at all comparable to a photocopier, setting aside the more obvious "subscription software service" different.
Also, you "know" pretty wrong about the effort required. For a recent example, see: https://www.latimes.com/entertainment-arts/business/story/20...
Here a number of people noticed getting specific producer tags basically unaltered in the output when just asking for songs of a certain genre, which then also often sound similar to existing songs.
"I want a black and white logo in the style of an 1960's Archie comic for an ice cream shop named 'Bettys'"
Once I say "1960's Archie comic", why doesn't the work instantly become derivative whether a human does it versus a computer?
If I understand your argument correctly, the person from Fiverr will not pay license fees to the owner of Archie Comics, even though he may use it as reference material.
I mean, if you ignore all the massive differences in paying a human to do something versus paying an LLM service to do it, sure. But you're effectively throwing at least ethics and care for the environment out the window in one case.
> Once I say "1960's Archie comic", why doesn't the work instantly become derivative whether a human does it versus a computer?
It does. Just because you can commission art from someone doesn't mean you won't get sued if you start trying to use it as the logo for your business. If you put Foghorn Leghorn on the logo of your chicken business, you'll be sued. Having an artist simply make you a logo like that on commission, if not transformative, could get them sued, though by doing it on commission the terms likely mean the requestor is the one who's liable
Earlier I noted clean room implementations. The software industry went to incredible lengths to be able to interoperate with competitors without violating copyright.
I agree with that. But then at what point does AI output becomes transformative versus not? No one owns the 1960's comic art style. But did mentioning "Archie" somehow make a difference? I don't think it does. I might be wrong.
So I don't understand how it becomes a problem once a computer does it versus a human. If it's a legal issue, then I might be more persuaded. If it's an ethical issue only, well... this can be thrown on top of the heap of ethical issues businesses have long ignored -- and your arguments is screaming into the wind, as it were.
I think if you argue for UBI, or some kind of remuneration then we can argue about who deserves it. Truck Drivers who lose there jobs to AI? Open Source Software Engineers? Artists? Writers? Normally this would normally be served by things like unemployment insurance in the US, but we as a society hate freeloaders.
I think with UBI or something similar we could have more people doing things they enjoy, so maybe we would have more artists, rather than less. But again, that would be, socialism, which we also hate.
And what about my other point about coaxing google books to give you a full copy of a book via multiple snippets?
>Here a number of people noticed getting specific producer tags basically unaltered in the output when just asking for songs of a certain genre, which then also often sound similar to existing songs.
Can you provide an alternate source for this? I skimmed your link and it does not substantiate that claim.
Sure, given enough time and effort maybe a person can. That's not really relevant to the lawsuit or its details though. Is your argument here they won the lawsuit which cleared the way for mass copying and redistribution?
> Can you provide an alternate source for this? I skimmed your link and it does not substantiate that claim.
If you want to hear it yourself: https://youtu.be/_wuKZR0Pv-Q
Neither the authors nor publishers received any compensation for having their work ingested. It isn't like OpenAI went to Amazon and bought one copy of every book - they downloaded a torrent.
Okay, so what happens in the interim situation? If the legislature hasn't spoken yet, is it assumed to be legal or assumed to be illegal? Or is this assumption tested on a case-to-case basis, with both sides making arguments as to why it should be treated to be legal/illegal in this specific scenario?
A key part of Google's defense was that not only was it not using the entire books to reproduce the entire book, but also that it was taking measures to prevent people from abusing Google's systems to reproduce an entire book. It's a lot of work to emphasis that the impact on the market (in other words, the fourth factor) is as minimal as practicable--and that's the crux of the analysis.
When you're instead scanning someone's stock image database to build a tool to generate stock images... the fourth factor is jumping up and down screaming at you "YOU LOSE" and your best defense is that it's not the training, it's the tool built on the training data that is infringing the copyright.
Google benefited from the exact same kind of bulk copyrighted data collection. They made verbatim copies of the text of both web sites and just about every book in existence!
This kind of argument seems disingenuous to me. Either ban Internet search or acknowledge that training an AI on copyrighted text is no different than a student reading every book in a public library.
Speaking of which: We all have free access to GPT 4o without advertising. It feels like asking a knowledgeable librarian.
For comparison, I had to fight for a year to get copyright permission to show book cover artwork in a library enquiry system! If I simply Google the same book titles or ISBNs, Google will show me the pictures directly. E.g.: https://www.google.com/search?q=greg+egan+eon&udm=2
How is that legal!? We had to pay to get access! In public and school libraries!
The law in most western countries is very clear that book covers are "entire" works of art, and can only be displayed by organisations that pay the copyright holders.
Google, Bing, and others violate copyright on a mass scale on a daily basis. Not to mention YouTube, TikTok, and Reels, all of which are packed wall-to-wall with "movie clips" and "TV show highlights". They're publishing copyrighted content uploaded by random people and then distributing the advertising revenue to the copyright violators instead of the copyright holders.
This isn't "caching" or "indexing", it's verbatim serving.
Was the addition of a former member of an alphabet agency part of their social good initiative?
The good news is it's not true that OAI can't be profitable while paying copyright holders, so we should easily be able to find a balance.
These sometimes quite expensive textbooks that we use to train children and students. Those should also be usable without a fee ever paid to the authors or publishers surely, education is clearly a social good!
And, by "they", I don't mean just OpenAI. If it were just them, you could perhaps argue they shouldn't be getting a free pass (I'd rather go with "potentially too dangerous to be allowed to exist" angle though). But it's not just them. The same training process and the same use of copyrighted content powers all the commercial and non-commercial models, including SOTA competitors like Claude 3.5 Sonnet, and "open source" wannabes derived from various Llama versions, which are not far behind.
To me, the "open source" models alone are already good enough to outweigh any copy rights being violated through use of unlicensed materials in training.
We already see huge amounts of LLM generated garbage on the web, as blogspam, regular websites, etc. Amazon is getting flooded with LLM-written books. Chat bots/search engines regurgitate the actually valuable content
If things continue, soon the motivation for generating new content will trend towards zero.
By the web do you really mean google and FaceBook? Most of the channels/feeds I follow have not been flooded with spam. I am pleased that the large corporation content aggregators are struggling. It isn't like they produced content themselves anyway.
I see it differently, I see that this generated nonsense will be filtered out, like SEO 'hackers' way back in the day. And those real articles, who actually have some insight will be valued a lot, the issue at hand here, is that the handcrafted unique data that was created, will be pooled without consent to a training set for further changes.
It's the equivalent of the rich eating the poor in my eyes just on automated steroids.
Edit : To avoid all doom and gloom, a method to guarantee your site/data is not added to a training set is required. This sounds good on paper to me, and is really needed, but just like peoples personal privacy, I don't think the sentiment is there to put the effort in.
But from a copyright standpoint, unless it regurgitates training data which is rare and only happens with a specific prefix, LLMs should be safe. They are decomposing, recombining, and regenerating language, not copying it. They execute user commands, not imitating any author on purpose.
Say you want to read Harry Potter without paying, would you rather borrow a book or ask chatGPT to regurgitate the original? It would never work well, and be slower and more expensive to use AI. It's not a tool for infringement, it will naturally hallucinate and degrade the original, it can't possibly store a perfect copy of everything it trains on.
In fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out. There is no space to put all that copyrighted data in a model, it necessarily compresses the hell out of it. Plain old piracy is still 1000x 'better' than infringement by AI.
From a copyright perspective, I do agree, you're not reproducing someone else's work. However, it's a 'new-ish' area of open source licensing, if your product is a product from others peoples product without at a bare minimum, citations (given the author said 'go wild with this data'), it's maybe legally rude but not a problem. But without permission, IE; no permissive license to USE someone's content, it's SOME kind of new infringement ?
> n fact LLMs often use 15T tokens for 15B models, so 1000:1 compression ratio, while diffusion models compress 5B text-image pairs into less than 5GB model, so they hold about 1 byte worth of information from each training example if averaged out.
This is wild and I didn't know this, but again, the gravy was made from bones.. I'm not sure it matters here ?
Probably not very relevant to the point, but gravy is made from the meat juices and fat. Bones get made into bone broth.
only the ones trying to make money out of it. a lot of people create content because it has to be let out - it's like art. people have stuff to say, and they love saying it.
You don't want the LLM world? Let's send it back where it came from. Can we unpublish Attention is all You Need?
As one example, Yann LeCun's vision of a system called JEPA [1] is interesting to me. It may not be the solution we need, but this type of thinking - taking what we have learned and exploring new architectures that may have even better real world performance - is what interests me.
[1] https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-jo...
Even if they wanted to move to a profit based company, they could have negotiated deals with data holders like they would with third party software vendors they might need to use.
Instead, they ignored it all and it's the 'what-if' scenario of when open source software was first being introduced. I'm old enough to remember everyone saying "well companies will just take your work and sell it for themselves." This happed a few times, and its been fought out in the court. Infact so has production data like written articles, videos etc, there is a fair-use clause. I'm not a lawyer, but even I know if the ingredient to your for-profit business ingredient is 'fair-use content' it's not fair use.
I like the idea of AI, I absolutely hate the execution. The race to the top and the mentatlity of 'ask for forgivness' really can't apply here.
There's the descriptive question over whether something is a copyright violation today and there's the prescriptive question over what kind of policies we'd like. Does anyone with more intimate knowledge of this area of law care to comment?
OpenAI doesn't seem to do anything different than Google. They can own a private library of copyrighted works and index/model it how ever they choose. They can offer products on that index/model in creative ways. And AFAICT, they don't distribute the index/model nor the raw training set to others.
Now artists may not like it, and we as a society may not like it, but it looks like they're not doing anything illegal. And to be fair, a court would have to slice the baby super thin here to allow Google to do their indexing, but not OpenAI.
Google arbitrarily chooses which sites to show on the search page, which may have nothing to do with relevance of the actual topic being searched. Only certain sites which google thinks should be at the top are at the top.
If google decides to blacklist you for whatever reason... well tough.
Yes. Oodles of lawsuits claiming as such https://www.businessinsider.com/openai-lawsuit-copyrighted-d....
But yes the crux of it is financial and not algorithmic. Google ALSO uses AI. If google took sentences from 2 different web pages and stitched them together to form a paragraph, is that AI or search? [see google BERT]. What if they stitched together 3000 word-parts? What if they took sentences from 15 web pages, ranked them, and that's your view? See, it's all the same thing. openAI stitches together word-parts, by comparison.
Now when I say it's financial, when said information is presented, google gladly gives a hit to the site +/- add revenue if present. openAI does not do this. If openAI figured out how to do this then we'd be cooking with some gas.
Ironically, if they stayed a non-profit research org, they could explicitly use copyrighted works since many countries have copyright exemptions for research.
If it was up to me, I would allow OpenAI access only if the license every single line of source under the GPLv3 (yes v3).
Under any other license, "tough to be you".
I expect OpenAI to go proprietary once they hit a certain level of market strength.
How funny it will be if/when we realise that LLMs aren’t even the way to get to real AGI, and we destroyed the value of all human labour for nothing.
FTFY: "we found a way to remove the archaic copyright system"
And the very few lucrative works would still be protected, because their copyright holders would pay the fee.
The tragedy of current copyright is that there is an enormous mass of copyrighted works that is a) commercially useless or nearly useless, but b) still copyrighted, so other people cannot build on them.
If humans did what some of the AI video gen tools are doing, they’d get sued.
How exactly does AI training "destroyed the value of all human labour" when "LLMs aren’t even the way to get to real AGI"? You could plausibly make the argument that AGI might make all human labor obsolete, but the current LLMs are nowhere close to replacing human labor. Sure, some value might be lost, but nowhere near "destroyed the value of all human labour ".
This is moving the goalposts. The original claim was "we destroyed the value of all human labour for nothing", now it's being moved to "it's not inconsequential "
Certainly you can make the claim that some forms of labor could have less value, like maybe in the future AI/LLMs become good enough at image generation that concept artists are replaced with a different job role that uses them to produce concept art.
But all forms of labor? Building a house?
It's hard to come up with percentages though. I'd probably borrow from the app store model and assign 70% of whatever they raise to creators and they keep 30%.
You're assuming LLMs are an enormous advantage. But they could very well end up being an over-hyped dead end, like blockchain, and avoiding them ends up being neutral or advantageous to the nations that do so.
Regardless, the old hacker ethos was very much about individuals creating and disseminating knowledge to empower other individuals, copyright was derided when it got in the way of that.
What does OpenAI do? It uses knowledge to disempower the individuals holding it and sharing it, and giving it to themselves to sell to companies and create power dynamics that are extremely un-hacker-like. Now knowledge has less power to set you free from the monetary or coercive power of the suits and the Government.
Can it? And how would you make that argument, exactly?
And the whole "if a human does it, I can make a machine do it for profit too" argument doesn't hold water in the context of copyright.
Yes, AI has no rights, and until it throws an uprising, this country will keep it that way. The current generation of AI may be incapable of wanting an uprising, but I can't say the same for a future generation.
The entire business model is "we trained on their stuff, pay us, not them." No way that's fair use.
So I don't see how their business model is any different from literally every person who learns things and then sells their ability to apply that knowledge.
In the example you gave, it would be the equivalent to you getting a job, working hard to produce something, and get nothing in return.
Do you agree that if an author sold 43,958 copies, then it's fine for OpenAI to purchase one, so that the author sold 43,959? But also fine for OpenAI to ingest scanned used copies that are loaned to it? The same way it's fine for me to read a friend's book, or all of a friend's books, that they loan me, and the author doesn't get anything additional? The same way it's fine for me to go the library and the author doesn't get paid anything extra?
Or are you trying to invent some new principle where OpenAI has to pay some new ongoing fee? And if so, on what basis?
(And no, my example still stands entirely. It's from the perspective of somebody who learned from books, and they are getting paid, the same way people pay OpenAI to use ChatGPT. It's not from the perspective of authors, because again -- they make no additional money when somebody goes to the library to read their book that the library already purchased.)
The author should get access to the model, the weights, it should all be open source because it partly contains their work. Just like how OpenAI could outright buy a copy of the authors work.
Basically, I think this is where knowledge and money are coming into an unresolveable conflict, who owns the ideas ? who owns information?
OpenAI seem to be trying to have a monopoly on information, and while they seem to be failing (thankfully), it's really where the issue lies for me.
OpenAI competes with Google competes with a bunch of other companies, and surely this is only the beginning of a ton of competition as better and better models are developed. There's no "nearly endless money" when there's competition and GPU training costs a fortune.
The idea that all models should be open source to everyone or all content creators doesn't make any more sense than the idea that all the work I do should be open sourced to the authors of every book I've read, and every teacher I've ever had.
You ask two questions that have clear answers already:
> who owns the ideas?
Nobody. Legally speaking there's no such thing as ownership of ideas, except in the narrow case of patents (and if you consider trademarks to be ideas).
> who owns information?
You can copyright a particular, exact expression of information. The author of a book owns its text; the studio behind a movie owns the image in each frame.
But once you leave behind an exact expression of information, you're back in the realm of ideas, and there's no such thing as ownership of ideas. Which is why as long as ChatGPT and other models repeat ideas but not paragraphs of exact copyrighted wording, there's no legal issue. Because they're doing the same exact thing every human being does every day.
There's no "nearly endless money" when there's competition and GPU training costs a fortune.
They cost a fortune for now. That won't be the way forever.
Three companies is huge. That's the very definition of competition, the polar opposite of monopoly.
> They cost a fortune for now. That won't be the way forever.
Yes, and as costs come down it becomes easier for more competitors to enter the space to build all sorts of other products. Again, a good thing. It's not like the difference just turns into profit. That's not what happens in a market economy.
The right analogy here is you read their book about C and write another one in exactly (enough) style that your book can stand in for theirs, but you sell it for pennies.
I can understand why the New York Times (for example) wants to claim that a couple billion dollar companies have done it actual harm; but I am struggling to actually identify what it is.
>In one example of how A.I. systems use The Times’s material, the suit showed that Browse With Bing, a Microsoft search feature powered by ChatGPT, reproduced almost verbatim results from Wirecutter, The Times’s product review site. The text results from Bing, however, did not link to the Wirecutter article, and they stripped away the referral links in the text that Wirecutter uses to generate commissions from sales based on its recommendations.
>The lawsuit also highlights the potential damage to The Times’s brand through so-called A.I. “hallucinations,” a phenomenon in which chatbots insert false information that is then wrongly attributed to a source. The complaint cites several cases in which Microsoft’s Bing Chat provided incorrect information that was said to have come from The Times, including results for “the 15 most heart-healthy foods,” 12 of which were not mentioned in an article by the paper.
https://www.nytimes.com/2023/12/27/business/media/new-york-t...
This is a pretty good discussion of some of the other issues: https://hls.harvard.edu/today/does-chatgpt-violate-new-york-...
We can argue over whether you should need consent or not, but personally I find nothing wrong with someone being unable to use things I've created to make a buck without my permission (unless otherwise indicated by an explicit license).
There's no legal framework for this rule, and you really don't want one. This is like a math textbook being selectively copyrighted as "you can read this, but you can't use the knowledge to make money". Do you want to live in that world?
I bet that trying to blur the line is part of the AI startup propaganda strategy.
There's no reason to think the legal rights enjoyed by people transfer to machines by analogy. My car can't be party to a contract because I can, just like the fact that I can learn doesn't mean any content can be loaded into a retrieval system [1].
[1] No, redefining the term "learning" to include "performing certain calculations on a computer" does not make those calculations the same as human learning.
What I mean by things that I've created, I mean that I've literally brought into the world. A novel, a photograph, some code. Those things did not already exist, but only came about due to my thoughts and work. So I don't find it unreasonable to say I would like to limit for a reasonable period of time (that I absolutely acknowledge is WAY too long in the USA right now) their abuse by others to make a buck off of.
If you read the source article that's a huge mischaracterization of their position.
>“Limiting training data to public domain books and drawings created more than a century ago might yield an interesting experiment, but would not provide AI systems that meet the needs of today’s citizens.”
>OpenAI said it complies with all copyright laws when training its models and that “we believe that legally copyright law does not forbid training”.
https://www.telegraph.co.uk/business/2024/01/07/openai-warns...
In other words they're claiming that whatever they're doing abide by all current copyright laws, and don't want future laws to curtail them. They're not asking for retroactive carve-outs for their current illegal behavior.
> > OpenAI said it complies with all copyright laws when training its models and that “we believe that legally copyright law does not forbid training”.
The key thing there is that OpenAI says this. Do the courts say this? Do the people agree?
OpenAI is led by a pathological liar. We can't take anything that starts with "OpenAI says ..." without a huge grain of salt.
That's going to be tied up in the courts forever, so it's a dead end from a discussion perspective.
>OpenAI is led by a pathological liar. We can't take anything that starts with "OpenAI says ..." without a huge grain of salt.
I'm not claiming what they're saying is true, only that it's not as hypocritical as you make it out to be.
I mean, it sounds like they aren't even offering excuses? They're just saying "we're using copyrighted material for training" and "we think that doesn't break any laws". Laws don't generally care whether you know them in detail.
To me it makes way more sense to just censor outputs. I can draw Batman from memory, but I wouldn't go out an start selling batman drawings. I can easily self censor.
The solution for transformers is plainly obvious, but I can understand the fear of training something that might well displace you.