Can't agree more. These AI systems have been built on the back of free labor for decades. We will look back on this and wonder how we essentially subsidized these behemoth corporations, then allowed them to extract subscription fees back from the very people it took the labor from.
Don't get me wrong, there are genuine innovations in AI and ML. But, on the same token, you can't have ChatGPT without content.
Yeah, I think, even at the most facial level, the output from an LLM or visual generative model is nothing like what you get as a search result, which is really only an index.
Maybe you can provide us with an argument as to why you think it's so obvious that they are similar?
Indexing content provides a useful service that content owners benefit from, too, so for a long time, there was definitely a mutual understanding between Google and site owners. I think the relationship has soured, somewhat coincidentally, in large part due to the way that Google has started to push against actually linking to sites, like by trying to provide inline answers and using AMP caches to avoid actually dropping the user out of Google's Search UI.
Personally, I think society is built on top of unspoken agreements that couldn't really be meaningfully codified, but then immensely harmed by those who exploit the resulting relationships that formed as a result of said unspoken agreements. The Internet's openness was once its biggest asset, but it feels like it quickly became the biggest liability almost overnight.
>Indexing content provides a useful service that content owners benefit from
I'm not sure about that. The Google Index put a lot of existing content creators out of business, like local newspapers. They certainly did not see this great benefit of a Google index. Even today, given the insanely low ad rates, because it's impossible to generate income from public content (hence the rise of paywalled subscription services) no content creator is making any money - and yet, Google is still worth a trillion dollars. And yes, there's also the fact that in the last decade or so, Google has started side-stepping the content providers by simply providing 'inline answers' and 'AMP caches'
So no, I don't see much difference between Generative AI and your 'traditional' aggregation services.
> The Google Index put a lot of existing content creators out of business, like local newspapers. They certainly did not see this great benefit of a Google index.
I dunno why you ignored the entire part of my comment wherein I describe the gradual fallout with Google, but frankly it's difficult to even begin with how infuriatingly wrong this line of thinking is.
So basically, you're saying that local newspapers, for example, did not see some great benefit from the Google search index. In fact, they're being killed by it! Shivers.
But wait a minute. How? How is Google killing the websites that its indexing? Is it because they're causing their hosting bills to skyrocket through negligent web crawling? Doesn't seem to be. Is it because they're charging websites through the nose to be listed? No, being listed in Google is free... Yeah, the reason why Google is "killing" local newspapers is simply local newspapers don't make as much money from Google traffic as they used to.
Forget about how Google is benefiting from local newspapers without giving back to them as much. That is happening, but there is a crucial and specific detail in your argument that needs to be scrutinized. If websites simply saw no benefit from being indexed by search engines, uhm, They Simply would choose to not be indexed in search engines. You can choose to do that. But they won't, because the problem isn't actually that Google is killing them. That's just the soppy rhetoric used to sell asinine ideas like the Link Tax wherein you will be billed for providing a hyperlink to a website.
The point I was going on about in my original comment was in fact that search indices like Google were in fact, always a give-and-take operation. Google benefits from having content in the index, websites benefit from the traffic they get from Google. The difference today is that Google does work harder to keep you off of the actual target website by adding features like info boxes containing snippets and again, AMP caches. However, note that these features don't really increase the "take": Google is taking exactly the same thing it always did, it's indexing your web sites and presenting its information in a transformed format on theirs. What it's giving, however, in terms of traffic to websites, has decreased over time.
Meanwhile, generative AI is a take operation. Certainly both search engines and generative AI models provide utility to the users of their respective services, but in case of generative AI models, it's only in service of the owners of the model, who give essentially nothing back, not credit, not a link, and certainly not revenue sharing of any kind.
This isn't in defense of Google's practices by any means: Like I said, they upset the balance that was in place from unspoken agreements. However, it definitely is in defense of the truth, and the truth is a phrase like "Google kills local newspapers" is bullshit, and if it wasn't, they'd all be running away from the Google index as fast as possible. Instead, many of them closely follow the latest SEO tricks, still.
And I hate this narrative anyways, because if organic Google traffic and Google Adsense was the only thing funding local newspapers, they were never sustainable by any measure in the first place.
Key differences are in crediting sources and where the revenue comes from:
- Google became a trillion-dollar company off advertising along side the search offering - the search offering itself linked back to sources and was fully available for $0.
- OpenAI is opaque with crediting sources and currently charges $20/mn for better output .
Google has suffered years of lawsuits that IIRC are still unresolved as a result of doing what OpenAI is doing (scraping the entire internet and using the content in their products).
That is (was) the issue when Google started providing answers along with the search results, answers which were sourced from somewhere else and whose author is not Google.
I don't think they should get away with that as it violates all of the safe harbor clauses from copyright law.
So, to answer your question, they're both doing bad things.
Google is unique from an internet platform perspective in the sense that the value of companies built on the platform are as big, or bigger than Google itself (probably, rough math here)
Yes, Google has captured massive value, but they've also generated massive amounts of value for companies (including startups!).
Can the same be said for OpenAI?
If they live up to their non-profit mission, I think it could.
> These AI systems have been built on the back of free labor for decades.
Do you repay publishers for information you’ve summarized or new insights you’ve gained after consuming their content? How is it different when AI does the same thing?
I paid for the newspaper, book copy or museum visit from which I learnt and got inspired. And like me, million of people did over the years. And yes, I also pirated part of that content as well.
You can't really compare machines with people when it comes to scale.
You can scale the volume of work for a machine in a way that would be impossible with people. No amount of Mechanical Turks could make content at the rate that Generative AI systems like ChatGPT, Dall-E, etc. can.
Treating content scraped from an original source without consent as fair use for training a machine somehow seems wrong. Especially since this AI system is then used to make money and that content is essential to its ability to do so, though without giving anything back to the original content creators.
Seems natural that any law regarding AI copyright should take these matters into account.
Are you implying normal humans get access to the data used in training GPts for free? Where can I read all those books for free, from an offline copy and without ads?
In a just world, royalties would be paid to those whose content AIs were trained on.
In reality, the most we'll likely see is a few people who successfully sue for their small piece of the pie, while the big companies carry on like nothing happened - further accelerating the transfer of wealth right to the top.
In that just world, the royalties would be pennies, just as is the case with Spotify. Creators would then endlessly complain that their content isn't worth more than it is.
It's fine if we shift to that model. The content creators won't be well compensated in that world, because their individual pieces of content are not in fact particularly valuable. It's a delusion that they are. What they really want is an artificially inflated over-compensation for their share (and if everyone got that, it wouldn't work at all).
And exactly as with Spotify, even with the payouts, the big corporation (OpenAI) will be the valuable thing in the end.
In a just world, publishers would be able to opt-out of AI training, and AI companies would have to respect this. Then, if they still want the material, they can approach publishers and buy the content. (Smaller publishers might make a "content pool" so they don't have to negotiate individually)
(This is how current search engines work, btw.. and that's why no one complains about them crawling content)
That might be misunderstanding the role of copyright as it applies to the worlds we live in.
Many here see copyright as a critical tool necessary for the existence of copy-left licenses, licenses that we use to make sure that what we write stays open in a way that others can benefit from.
We rely on copyright to ensure that our code stays “Free as in speech” not simply “Free as in beer”. Simply accepting no form of creator control is at odds with our desire to make sure that the things we produce stay open for others to use since there is nearly no incentive for the next person to keep our code open.
>In the same way a writer reads books and use that knowledge to create more, an ai model can do the same.
No, it can't. Literally.
The AI model doesn't decide which data goes into training it. People do. These people are doing something very, very bad.
Also, the AI model can't be held responsible for its actions. Again, people that create it and use it can be, however.
People on HN like to point out that an AI model is "just a tool", so work created with AI is still the work of the user.
It's only reasonable to point out that a tool is not the same thing as a writer. A tool lacks agency. Therefore, a tool isn't "learning" just like a writer.
A ZIP file isn't "learning" when it's created by transforming data into a binary file. Neither does a JPEG file.
You can't have your cake and eat it too. If the AI model is equivalent to a sentient human being in your eyes, it should have the same rights and responsibilities. As long as it's being used, the equivalence isn't there.
I think we need something like copyright exemptions premised on an LLM equivalent of GPL. You can train the model on whatever, but if it includes copyrighted content, then you either have to release everything fully open (training data, code, weights) or pay to license the training data. Likewise, any fine-tunes used in a commercial setting inherit the same license and requirements. And outputs can’t be copyrighted.
This will of course never happen because of AI’s part in the new Cold War with China, but it seems the only way to equitably resolve all the theft and running roughshod over other people’s work that the current landscape twists itself into knots trying to justify.
"Of course, we might fail. If that happens, well, we can actually see what that future looks like. Cory Doctorow, blogger and science fiction writer, coined the pungent word, "Enshittification" for it. By this, he means the fall off in the quality of online sites and information."
But I wonder why newsrooms do not see the potential silver lining in the long-run? If the open Web becomes an AI-fueled garbage bin, as it certainly seems, it would seem reasonable to expect that the value of quality sources of information would increase and not decrease, provided that these sources are either technically or judicially blocked from feeding into the AIs.
It seems obvious to me that a system in which intellectual property can be ingested into an artificial intelligence system and then provide no value to the original creator is unsustainable.
The value of content that is generated through real-world effort like research and investigation will plummet towards zero. And then what? You can't just have a society built off of LLMs feeding each other.
It also feels obvious that it's going to be very hard to enforce any kind of rules around this.
And further obvious that our current legal framework was not built with this kind of situation in mind, and that any decisions around rules right now are going to have an outsized impact on the path we take into the future.
So what now? Are we going to let some panel of judges rule on whether training violates copyright, and call it a day?
We're going to need some good, nuanced, well-intentioned debate and come up with a coherent set of rules that to create a fair and sustainable system.
No, we should calibrate expectations accordingly and not assume that even the signatories to the rules won’t rush to exploit the loopholes the nanosecond they put their signatures there.
There should be a way to codify what “the spirit” of the rules is and that seeking a way to subvert it is against the rules too.
Technically it's not a copy. And nothing was stolen. It's a best fit curve among a series of datapoints. The datapoint is a copy, if the best fit curve never touches the datapoint then it's technically not a copy.
The difficult part is the technicality here is legal and ethical from any standpoint. The high level ramifications are a bit unfair in the sense that yes the data is being used to create an AI that can replace you and your job from data you created to DO your job.
This is the conundrum with AI. Our legal system and moral intuition makes it permissible to read and interpret things which is pretty much all ML does. Does it make sense to make it illegal for someone to learn programming for free from online articles? No. Does it make sense to make it illegal for someone to learn programming for free from online articles to take over your job? No. Should it be illegal to use tools to help me learn programming for free by using tools to help me read for free from online articles so I can take over your job? No.
But if that tool is AI, suddenly, Yes, it should be illegal. The logic makes no sense. But we shouldn't rely on logic to maintain our morals, even if it doesn't hold logical cohesion if the outcomes are negative it should still be immoral imo.
Because music has a lot of additional law written giving additional protections to song-writers independent of performers and recordings. That gives the abstract tonal sequence it's own copyright.
I think it's an apt analogy, though I disagree about the implication.
If I use ChatGPT to create a work, and that work is "close enough" to an existing copyrighted work, then it seems like I am guilty of copyright violation, not ChatGPT.
It's not an analogy. This is actually what is done with ML. It is literally a best fit curve problem.
Or maybe it is actually an analogy, but then if this was the case the entire field of ML is capable of only understanding the intricacies of ML through the analogy of curve fitting and what's actually going on underneath the analogy remains elusive.
True, but most points on the curve aren't close to Any data point. That means most of the output of ML is completely original. Let's use a simplified example of a straight line between two datapoints. Example:
Point A ------|----------------------------------------|------Point B
For a line segment (above) between two points, most of the line does not approach the vicinity of Either point (the boundary of closeness for points A and B is demarcated figuratively with a pipe "|" if the line segment is between the pipe and the point it is too close, if it is not then it is an original work). This intuition still applies even if the line only moves close to the point and does not touch either point. Basically the output of ML is by majority not even close to a copy as most of the curve is far from any point.
The only way for most of the line to be a copy is if the data cluster is so close and similar that the data itself is mostly a chain of "similar" copies. Not sure if you're catching my meaning here. Example:
Point A --|--|-- Point B
Above A and B cross their own thresholds and are essentially "close enough" copied data points. The left Pipe is the threshold for B and the right pipe is the threshold for A. As a result the entire line between the two points must be a an illegal "close enough" copy as well.
If such data is used it means existing data is in violation of copywrite law already. The logical implication is this:
For most of the results of ML to be a technical "close enough" copy of the datapoints, you must also admit that most of your data contained "close enough" copies as well.
As a side note this kind of thing can be useful for defining a quantitative measure of what close enough even means as we can certainly define a numeric threshold between close enough and not close enough for copywrite law.
I really hate this article. They compared training ChatGPT to actual copy&paste sites without any supporting argument, complained about how much money OpenAI is making without paying them, and finished with blaming generative AI for the declining quality in Google search results.
Of course there is fair use, etc etc. But modern copyright was a reaction to the printing press so that people were motivated to still create content in the face of new technology.
If current copyright laws do not protect people from creating something new because of fear that they will be ripped off, then new copyright laws need to come.
One thing I wondered about how arguments from one side of the issue say that AI copying and extracting information for free isn't stealing, but what if you use their argument against things that aren't copyright like secrets, military, corporate, trade secrets.
Like can if an LLM saw the coca-cola forumla and the weights are released, what are the consequences? If it ingested top secret confidential information and released the weights, I assume that counts as stealing something and distributing it.
The (in my view) problem with the author's argument is that the first step he claims is happening, is not. Publicly available content gets read, as is the point of publicly publishing it. Then the user uses a computer program to make some statistics about the bit of content. Those bits of statistics about that specific work, on their own, cannot reproduce or recrate the specific work. Then those statistics are put into a database and combined with the stats about billions of other works. Then another program is written to query the database to make probabilistic guesses responding to the prompts from a user. It's this last stage could potentially recreate a work in an infringing manner. But everthing that led up to that point (creating the model) is simply not something that current law considers to be infringing of copyright in any meaningful way. It doesn't even require a "fair use" assessment, because, creating statistics about a work, that cannot on their own reproduce the work, does not create a copy, nor does it make a public performance of the work.
Is this all terribly unfair to the people that published their work assuming this couldn't happen? Yes. But the response needs to be "lets come up with and pass better law" and not "lets twist and contort the current law to be something it's not."
Why should a crime depend on how the tool used for the crime was created.
Like 2 guys write 2 scripts that output a copyrighted poem, then the cool one guy, call him Sam can go free because he used algorithm A but second guy goes to jail because he used algorithm B where the crime is exactly identical.
Forgive me for this one, but it comes from genuine curiosity and not snark. You are making assertions about how copyright law works, but you don't qualify it with either IANAL or any legal credentials. So I must ask: do you have a basis for these claims?
I love participating in armchair analysis of the law, since in software we pretty much have no choice but to do so anyway, but my understanding has always been that we still don't actually have strong case-law for machine learning and AI. It does seem like the existing cases regarding weights and ML training have leaned strongly towards the weights in general not being considered a derivative work, but I have doubts that the law would see this as black and white; for example, even if the general consensus is that ML training to produce weights, in and of itself, does not create a derivative work, if you are able to show that a given set of weights is able to verbatim reproduce inputs (as a result of overfitting or memorization), I have my suspicions that it would not be shrugged off so easily. In true "color of my bits" fashion, I think that from a legal standpoint, the actual technical means by which something was accomplished doesn't matter if the process as a whole is effectively copyright infringement.
There do seem to be some ongoing cases regarding this such as Getty Images v. Stability AI and it will be interesting to see their result.
> I think that from a legal standpoint, the actual technical means by which something was accomplished doesn't matter if the process as a whole is effectively copyright infringement.
Which is why when the user of the model prompts for something infringing, and is successful at getting close to verbatim output (because the prompt was too constraining, becuase the work is overrepresented in the training) it is that particular output that is infringing. And maybe that means that services operating that prompt/response software are guilty of contributory infringment if they can't adequetly prevent that kind of output.
But that doesn not mean that training the model was infringing. Nor does that mean distribution of the model is infringing. And if a user of the prompt/response software never prompts for anything infringing, and the software never spontaneously recreates anything infringing, there's no infringment happening.
There are lots of technologies out there that are highly capable of enabling infringment at a massive scale. And where the vast majority of their actual usage is absolutely infringing. But we don't completely shut down those technologies that on their own - are not infringing. Bittorrent clients are pefectly legal to develop. And distribute. And people use those clients to commit infringment at large scale. But they are still pefectly legal to write and distrubute.
TensorFlow is also perfectly legal to develop and distribute, and no one contests this.
People object to specific artifact, "model weights", which were produced using copyrighted works at the input, and can be used to reproduce those same copyrighted works back. In bittorrent analogy, people want to shut down specific pirate trackers and the pirate bay website.
> First, a derivative work still has to be “substantially similar” to the original in order to be infringing. If the original is transformed or abridged or adapted to such an extent that this is no longer true, then it’s not a derivative work. A 10-line summary of a 15,000-line epic isn’t a derivative work, and neither are most summaries of books that people make in order to describe those copyrighted works to others.
The statistics generated about the works entered as input, do not resemble the original works. Nor can those statistics on their own reproduce the original work. At most they are brief mathematical summaries of the work. And it's only after combining those stats with the stats of billions of other works (which is its own creative process to determine the best statistical methodologies to achieve that combination) that anything intelligble can be produced in the output stage.
I think the case for Stable Diffusion in general is not too bad, however EFF tempers their optimism when it comes to cases where the model may actually memorize the inputs:
> To sum up: a diffusion model can, in rare circumstances, generate images that resemble elements of the training data. De-duplication can substantially reduce the risk of this occurring. But the strongest copyright suit against a diffusion-based AI art generator would likely be one brought by the holder of the copyright in an image that subsequently was actually reproduced this way.
EFF's position seems to be (to which I personally agree, FWIW) that Stable Diffusion almost certainly does not run afoul of at least the vast majority of copyright holders of data it was trained from.
> The statistics generated about the works entered as input, do not resemble the original works. Nor can those statistics on their own reproduce the original work. At most they are brief mathematical summaries of the work.
Of course, this needs a lot of qualification. Compression and intelligence are generally considered to be related, and indeed, compression also works on statistical analysis (like entropy coding a la Huffman, or frequency analysis via Fourier transforms). Granted, compression algorithms are designed to reproduce their input verbatim--it's the entire point. But I think ML weights may exist somewhere "in the middle" so to speak; depending on the model architecture and how it's trained, it may be more or less literally like compression. Vastly overfit models are very much like compression, whereas large generalized models like Stable Diffusion are pretty far away and yes, generally can't reproduce inputs verbatim. (However: I suspect many LoRA models are quite overfit and may not be in the same boat.)
However, that's just for image generation. I feel like LLMs and text generation are an entirely different ballgame, and given that we can't actually inspect the model weights in the case of GPT4, the best we can really do to surmise what's going on is to see how badly it seems to overfit its training data.
I am unconvinced that this matter is settled as a whole, although I do think the EFF article presents a good overview of the case regarding Stable Diffusion and it does coincide pretty closely with what I actually believe. But this article is about large language models, which may legitimately be a completely different ball game.
One thing that I think people forget about is that the prompt used when "reproduc[ing] those same copyrighted works" is also a part of why it spits out similar things. It's not just the model doing it. A traditional artist can be prompted to recreate a copyrighted work in much the same way with the right prompts.
I don't think most people are misinterpreting things. The truth is that models which are not terribly overfit literally don't output verbatim inputs often, in fact, for Stable Diffusion it's apparently nearly infinitesimally small odds, and this is good because that implies that the weights are in fact, not literally encoding some crazy kind of compressed copies of the images in question.
On the other hand, if you prompt a code generating model with some comment and a function declaration that it knows exists and it spits out 100+ lines of nearly verbatim code, that's a completely different story entirely. If I prompt a human with that sort of thing, they will almost certainly write different code even if they've seen the original source code in question. This is in part because the way humans write code is different from the way LLMs write code; humans tend to iterate somewhat non-linearly, and I think if you ask the same person to write the same thing on different days, they would probably come up with different results. It would be quite rare for a human to just see a familiar segment of code and then begin dumping near-verbatim copies of existing codebases.
AI models that readily and easily bias themselves toward outputting their inputs do exist. It is not clear how many models actually do this, but this is definitely a huge part of the concern when people talk about copyright and model weights.
It's a bit clouded by people who are just generally hoping that today's AI model weights are illegal for social reasons, but that's not the position I am trying to present. (I'm not really sure what we should do regarding societal impact.)
It's an interesting read and makes a good case for why none of the steps are directly copyright infringement, even if you can prompt the output to be (and in that case the person doing the prompting should be the one at fault, same as someone drawing something infringing directly).
It talks about copyright infringement in music, but it applies just as well to AI training, just substitute "scrambled file" with "model weights":
> The scrambled file still has the copyright Colour because it came from the copyrighted input file. It doesn't matter that it looks like, or maybe even is bit-for-bit identical with, some other file that you could get from a random number generator. It happens that you didn't get it from a random number generator. You got it from copyrighted material; it is copyrighted. The randomly-generated file, even if bit-for-bit identical, would have a different Colour. The Colour inherits through all scrambling and descrambling operations and you're distributing a copyrighted work,
They featured a naked, burned girl on the front page of the New York Times to sell copies. I am not aware of any compensation the New York Times offered her. Her permission was never asked.
That picture has defined how the public has viewed her for the rest of her life.
They would argue that doing this is in furtherance of public goods such as people knowing what is going on and they could not realistically seek permission from every person much less compensate every person featured in a news photo or story.
I think that may be a valid argument. But doesn't AI also have the potential to benefit society. Isn't using news articles as input for AI making AI better and more useful to society. If the news can use people's images and suffering to benefit society (while making money), why can't their news stories be used to train AI to benefit even more people with knowledge?
I don't think it should be ok to copy the literal works, the example of it emitting large chunks of articles from NYT is egregious, but it should be ok for it to learn and synthesize anything it can see. It should ideally, for producers and consumers of information, also be able to function as an index.
Some of the pushback is the desire to collect fees or social capital for works, which is understandable, but at the same time that inherently creates walled gardens of information and the ability to combine (synthesize), something which could be considered outdated compared to what these algorithms could offer.
One somewhat reasonable outcome could be that for profit offerings must develop ways to limit excerpts and acknowledge sources, the latter similarly to how Google search works. I think it would be a terrible outcome if knowledge and AI abilities were splintered off to different product groupings (eg NYT only available via Apple AI).
It is easy to check for literal copies of the training material. I expect the providers to just add a filter doing that.
But I don’t think it resolves all the problems. Newspapers are losing their business model. Maybe it is just creative destruction and we’ll have something better in their place. Maybe, but journalism is quite important in our societies.
This argument, like everything else, is meaningless unless we talk about scale. It's like saying, well, I don't mind if someone sits by the road with pen and paper writing down license plate numbers.
But now, when the scenario is adjusted to be a network of ALPN sensors blanketing an entire metro area, am I just "purposely misleading to try and cripple scary new tech" when I object to such a system?
I totally get that the scale is massive. But I am failing to see what copyright has to do with it.
I understand that AI is capable of copyright infringement, I too can draw a batman symbol off the top of my head, and it seems the fix for this simply output filtering AI content. But look at any AI-copyright complaint and all they focus on is training.
Adjusting weights based on viewing data also describes a JPEG file, if you squint. "I didn't copy a picture, I merely adjusted cosine transform weights after viewing it."
Since you asked for someone to explain: IMO you're focusing on a mechanistic explanation of how ML happens, but if you take only a step back, the resulting models are really similar to a compressed blob of the training data. At times they regurgitate the training data almost exactly. This is what most people are seeing and (IMO, rightly) calling out as blatant infringement.
Slightly different topic: I think calling it "learning" is brilliant PR, but if you survey AI experts, they'll mostly agree it's not learning in the same way a human brain learns. You and I get more legal leeway than a computer, and it'll be interesting how that works out if/when AGI becomes a thing, but we're nowhere near there yet.
Usually, you pay what they ask and, if you don't like the price, you don't use their work. It's very simple, but OpenAI skipping that step opens them up to courts deciding that price, if a price is actually owed. That price would probably be more than 0, which is what the authors have gotten so far.
(2) if OpenAI still wants to use material in training, the can approach author with an offer.
Something like Google is a win-win proposition: Google gets useful results, while publisher gets clicks and ad impressions. But being included into AI training is not like that, authors get nothing for being included in the AI training set. So they should be free to charge whatever amount of money they want, or opt out completely.
Look, I publish (nearly) all my blog posts for _free_ on the web because I like doing that, and I _still_ don't want chatbots scraping my content to serve their algos (yes, I'm blocking them now via robots.txt) because the reason I put content out for free is partially because I want my name and my expertise out there. I'm part of a community of authors if you will. Call it selfish or self-serving if you will, but that's my right as the author.
Having stuff I've contributed to the internet get ingested into a model and regurgitated out such that my contributions are sidelined…well, that's just not acceptable, any more than a person taking my blog post, rewriting it with a handful of other brief sources, then republishing it under their name with no attribution back to me is acceptable.
I'm a bit dubious about all these complaints. My snarky side wants to respond to
> By OpenAI's logic, any work you put online is fair game to be swiped and incorporated into the company's large language models.
With
> Any work you put online is fair game to be swiped and incorporated into a human's brain.
It's tough because the ability to copy an LLM is feasible, and an LLM is much more likely to be able to reproduce the underlying work with high fidelity, but there's no law that says that someone with a photographic memory can't look at your site data.
78 comments
[ 2.3 ms ] story [ 131 ms ] threadDon't get me wrong, there are genuine innovations in AI and ML. But, on the same token, you can't have ChatGPT without content.
In practical terms, is that different from Google becoming a trillion-dollar company from its search offering?
Maybe you can provide us with an argument as to why you think it's so obvious that they are similar?
Google directs people to the people that created content and their platforms, allowing them to profit off of their labor.
ChatGPT completely obscures the source.
Simple enough, it seems.
Personally, I think society is built on top of unspoken agreements that couldn't really be meaningfully codified, but then immensely harmed by those who exploit the resulting relationships that formed as a result of said unspoken agreements. The Internet's openness was once its biggest asset, but it feels like it quickly became the biggest liability almost overnight.
I'm not sure about that. The Google Index put a lot of existing content creators out of business, like local newspapers. They certainly did not see this great benefit of a Google index. Even today, given the insanely low ad rates, because it's impossible to generate income from public content (hence the rise of paywalled subscription services) no content creator is making any money - and yet, Google is still worth a trillion dollars. And yes, there's also the fact that in the last decade or so, Google has started side-stepping the content providers by simply providing 'inline answers' and 'AMP caches'
So no, I don't see much difference between Generative AI and your 'traditional' aggregation services.
I dunno why you ignored the entire part of my comment wherein I describe the gradual fallout with Google, but frankly it's difficult to even begin with how infuriatingly wrong this line of thinking is.
So basically, you're saying that local newspapers, for example, did not see some great benefit from the Google search index. In fact, they're being killed by it! Shivers.
But wait a minute. How? How is Google killing the websites that its indexing? Is it because they're causing their hosting bills to skyrocket through negligent web crawling? Doesn't seem to be. Is it because they're charging websites through the nose to be listed? No, being listed in Google is free... Yeah, the reason why Google is "killing" local newspapers is simply local newspapers don't make as much money from Google traffic as they used to.
Forget about how Google is benefiting from local newspapers without giving back to them as much. That is happening, but there is a crucial and specific detail in your argument that needs to be scrutinized. If websites simply saw no benefit from being indexed by search engines, uhm, They Simply would choose to not be indexed in search engines. You can choose to do that. But they won't, because the problem isn't actually that Google is killing them. That's just the soppy rhetoric used to sell asinine ideas like the Link Tax wherein you will be billed for providing a hyperlink to a website.
The point I was going on about in my original comment was in fact that search indices like Google were in fact, always a give-and-take operation. Google benefits from having content in the index, websites benefit from the traffic they get from Google. The difference today is that Google does work harder to keep you off of the actual target website by adding features like info boxes containing snippets and again, AMP caches. However, note that these features don't really increase the "take": Google is taking exactly the same thing it always did, it's indexing your web sites and presenting its information in a transformed format on theirs. What it's giving, however, in terms of traffic to websites, has decreased over time.
Meanwhile, generative AI is a take operation. Certainly both search engines and generative AI models provide utility to the users of their respective services, but in case of generative AI models, it's only in service of the owners of the model, who give essentially nothing back, not credit, not a link, and certainly not revenue sharing of any kind.
This isn't in defense of Google's practices by any means: Like I said, they upset the balance that was in place from unspoken agreements. However, it definitely is in defense of the truth, and the truth is a phrase like "Google kills local newspapers" is bullshit, and if it wasn't, they'd all be running away from the Google index as fast as possible. Instead, many of them closely follow the latest SEO tricks, still.
And I hate this narrative anyways, because if organic Google traffic and Google Adsense was the only thing funding local newspapers, they were never sustainable by any measure in the first place.
- Google became a trillion-dollar company off advertising along side the search offering - the search offering itself linked back to sources and was fully available for $0.
- OpenAI is opaque with crediting sources and currently charges $20/mn for better output .
I don't think they should get away with that as it violates all of the safe harbor clauses from copyright law.
So, to answer your question, they're both doing bad things.
Yes, Google has captured massive value, but they've also generated massive amounts of value for companies (including startups!).
Can the same be said for OpenAI?
If they live up to their non-profit mission, I think it could.
But it's not clear that it will.
Do you repay publishers for information you’ve summarized or new insights you’ve gained after consuming their content? How is it different when AI does the same thing?
You can scale the volume of work for a machine in a way that would be impossible with people. No amount of Mechanical Turks could make content at the rate that Generative AI systems like ChatGPT, Dall-E, etc. can.
Treating content scraped from an original source without consent as fair use for training a machine somehow seems wrong. Especially since this AI system is then used to make money and that content is essential to its ability to do so, though without giving anything back to the original content creators.
Seems natural that any law regarding AI copyright should take these matters into account.
In reality, the most we'll likely see is a few people who successfully sue for their small piece of the pie, while the big companies carry on like nothing happened - further accelerating the transfer of wealth right to the top.
It's fine if we shift to that model. The content creators won't be well compensated in that world, because their individual pieces of content are not in fact particularly valuable. It's a delusion that they are. What they really want is an artificially inflated over-compensation for their share (and if everyone got that, it wouldn't work at all).
And exactly as with Spotify, even with the payouts, the big corporation (OpenAI) will be the valuable thing in the end.
(This is how current search engines work, btw.. and that's why no one complains about them crawling content)
In the same way a writer reads books and use that knowledge to create more, an ai model can do the same.
How would writers do this if copyright is stifling as you say?
Once you make some, you kinda want to have some control over it.
Many here see copyright as a critical tool necessary for the existence of copy-left licenses, licenses that we use to make sure that what we write stays open in a way that others can benefit from.
We rely on copyright to ensure that our code stays “Free as in speech” not simply “Free as in beer”. Simply accepting no form of creator control is at odds with our desire to make sure that the things we produce stay open for others to use since there is nearly no incentive for the next person to keep our code open.
No, it can't. Literally.
The AI model doesn't decide which data goes into training it. People do. These people are doing something very, very bad.
Also, the AI model can't be held responsible for its actions. Again, people that create it and use it can be, however.
People on HN like to point out that an AI model is "just a tool", so work created with AI is still the work of the user.
It's only reasonable to point out that a tool is not the same thing as a writer. A tool lacks agency. Therefore, a tool isn't "learning" just like a writer.
A ZIP file isn't "learning" when it's created by transforming data into a binary file. Neither does a JPEG file.
You can't have your cake and eat it too. If the AI model is equivalent to a sentient human being in your eyes, it should have the same rights and responsibilities. As long as it's being used, the equivalence isn't there.
This will of course never happen because of AI’s part in the new Cold War with China, but it seems the only way to equitably resolve all the theft and running roughshod over other people’s work that the current landscape twists itself into knots trying to justify.
But I wonder why newsrooms do not see the potential silver lining in the long-run? If the open Web becomes an AI-fueled garbage bin, as it certainly seems, it would seem reasonable to expect that the value of quality sources of information would increase and not decrease, provided that these sources are either technically or judicially blocked from feeding into the AIs.
The value of content that is generated through real-world effort like research and investigation will plummet towards zero. And then what? You can't just have a society built off of LLMs feeding each other.
It also feels obvious that it's going to be very hard to enforce any kind of rules around this.
And further obvious that our current legal framework was not built with this kind of situation in mind, and that any decisions around rules right now are going to have an outsized impact on the path we take into the future.
So what now? Are we going to let some panel of judges rule on whether training violates copyright, and call it a day?
We're going to need some good, nuanced, well-intentioned debate and come up with a coherent set of rules that to create a fair and sustainable system.
There should be a way to codify what “the spirit” of the rules is and that seeking a way to subvert it is against the rules too.
The difficult part is the technicality here is legal and ethical from any standpoint. The high level ramifications are a bit unfair in the sense that yes the data is being used to create an AI that can replace you and your job from data you created to DO your job.
This is the conundrum with AI. Our legal system and moral intuition makes it permissible to read and interpret things which is pretty much all ML does. Does it make sense to make it illegal for someone to learn programming for free from online articles? No. Does it make sense to make it illegal for someone to learn programming for free from online articles to take over your job? No. Should it be illegal to use tools to help me learn programming for free by using tools to help me read for free from online articles so I can take over your job? No.
But if that tool is AI, suddenly, Yes, it should be illegal. The logic makes no sense. But we shouldn't rely on logic to maintain our morals, even if it doesn't hold logical cohesion if the outcomes are negative it should still be immoral imo.
There have been many cases in music where the offending song was forced to pay because it was "close enough" to the curve but not touching it.
If I use ChatGPT to create a work, and that work is "close enough" to an existing copyrighted work, then it seems like I am guilty of copyright violation, not ChatGPT.
Or maybe it is actually an analogy, but then if this was the case the entire field of ML is capable of only understanding the intricacies of ML through the analogy of curve fitting and what's actually going on underneath the analogy remains elusive.
The only way for most of the line to be a copy is if the data cluster is so close and similar that the data itself is mostly a chain of "similar" copies. Not sure if you're catching my meaning here. Example:
Above A and B cross their own thresholds and are essentially "close enough" copied data points. The left Pipe is the threshold for B and the right pipe is the threshold for A. As a result the entire line between the two points must be a an illegal "close enough" copy as well.If such data is used it means existing data is in violation of copywrite law already. The logical implication is this:
For most of the results of ML to be a technical "close enough" copy of the datapoints, you must also admit that most of your data contained "close enough" copies as well.
As a side note this kind of thing can be useful for defining a quantitative measure of what close enough even means as we can certainly define a numeric threshold between close enough and not close enough for copywrite law.
If current copyright laws do not protect people from creating something new because of fear that they will be ripped off, then new copyright laws need to come.
One thing I wondered about how arguments from one side of the issue say that AI copying and extracting information for free isn't stealing, but what if you use their argument against things that aren't copyright like secrets, military, corporate, trade secrets.
Like can if an LLM saw the coca-cola forumla and the weights are released, what are the consequences? If it ingested top secret confidential information and released the weights, I assume that counts as stealing something and distributing it.
Is this all terribly unfair to the people that published their work assuming this couldn't happen? Yes. But the response needs to be "lets come up with and pass better law" and not "lets twist and contort the current law to be something it's not."
I love participating in armchair analysis of the law, since in software we pretty much have no choice but to do so anyway, but my understanding has always been that we still don't actually have strong case-law for machine learning and AI. It does seem like the existing cases regarding weights and ML training have leaned strongly towards the weights in general not being considered a derivative work, but I have doubts that the law would see this as black and white; for example, even if the general consensus is that ML training to produce weights, in and of itself, does not create a derivative work, if you are able to show that a given set of weights is able to verbatim reproduce inputs (as a result of overfitting or memorization), I have my suspicions that it would not be shrugged off so easily. In true "color of my bits" fashion, I think that from a legal standpoint, the actual technical means by which something was accomplished doesn't matter if the process as a whole is effectively copyright infringement.
There do seem to be some ongoing cases regarding this such as Getty Images v. Stability AI and it will be interesting to see their result.
Which is why when the user of the model prompts for something infringing, and is successful at getting close to verbatim output (because the prompt was too constraining, becuase the work is overrepresented in the training) it is that particular output that is infringing. And maybe that means that services operating that prompt/response software are guilty of contributory infringment if they can't adequetly prevent that kind of output.
But that doesn not mean that training the model was infringing. Nor does that mean distribution of the model is infringing. And if a user of the prompt/response software never prompts for anything infringing, and the software never spontaneously recreates anything infringing, there's no infringment happening.
There are lots of technologies out there that are highly capable of enabling infringment at a massive scale. And where the vast majority of their actual usage is absolutely infringing. But we don't completely shut down those technologies that on their own - are not infringing. Bittorrent clients are pefectly legal to develop. And distribute. And people use those clients to commit infringment at large scale. But they are still pefectly legal to write and distrubute.
People object to specific artifact, "model weights", which were produced using copyrighted works at the input, and can be used to reproduce those same copyrighted works back. In bittorrent analogy, people want to shut down specific pirate trackers and the pirate bay website.
> First, a derivative work still has to be “substantially similar” to the original in order to be infringing. If the original is transformed or abridged or adapted to such an extent that this is no longer true, then it’s not a derivative work. A 10-line summary of a 15,000-line epic isn’t a derivative work, and neither are most summaries of books that people make in order to describe those copyrighted works to others.
The statistics generated about the works entered as input, do not resemble the original works. Nor can those statistics on their own reproduce the original work. At most they are brief mathematical summaries of the work. And it's only after combining those stats with the stats of billions of other works (which is its own creative process to determine the best statistical methodologies to achieve that combination) that anything intelligble can be produced in the output stage.
> To sum up: a diffusion model can, in rare circumstances, generate images that resemble elements of the training data. De-duplication can substantially reduce the risk of this occurring. But the strongest copyright suit against a diffusion-based AI art generator would likely be one brought by the holder of the copyright in an image that subsequently was actually reproduced this way.
EFF's position seems to be (to which I personally agree, FWIW) that Stable Diffusion almost certainly does not run afoul of at least the vast majority of copyright holders of data it was trained from.
> The statistics generated about the works entered as input, do not resemble the original works. Nor can those statistics on their own reproduce the original work. At most they are brief mathematical summaries of the work.
Of course, this needs a lot of qualification. Compression and intelligence are generally considered to be related, and indeed, compression also works on statistical analysis (like entropy coding a la Huffman, or frequency analysis via Fourier transforms). Granted, compression algorithms are designed to reproduce their input verbatim--it's the entire point. But I think ML weights may exist somewhere "in the middle" so to speak; depending on the model architecture and how it's trained, it may be more or less literally like compression. Vastly overfit models are very much like compression, whereas large generalized models like Stable Diffusion are pretty far away and yes, generally can't reproduce inputs verbatim. (However: I suspect many LoRA models are quite overfit and may not be in the same boat.)
However, that's just for image generation. I feel like LLMs and text generation are an entirely different ballgame, and given that we can't actually inspect the model weights in the case of GPT4, the best we can really do to surmise what's going on is to see how badly it seems to overfit its training data.
I am unconvinced that this matter is settled as a whole, although I do think the EFF article presents a good overview of the case regarding Stable Diffusion and it does coincide pretty closely with what I actually believe. But this article is about large language models, which may legitimately be a completely different ball game.
On the other hand, if you prompt a code generating model with some comment and a function declaration that it knows exists and it spits out 100+ lines of nearly verbatim code, that's a completely different story entirely. If I prompt a human with that sort of thing, they will almost certainly write different code even if they've seen the original source code in question. This is in part because the way humans write code is different from the way LLMs write code; humans tend to iterate somewhat non-linearly, and I think if you ask the same person to write the same thing on different days, they would probably come up with different results. It would be quite rare for a human to just see a familiar segment of code and then begin dumping near-verbatim copies of existing codebases.
AI models that readily and easily bias themselves toward outputting their inputs do exist. It is not clear how many models actually do this, but this is definitely a huge part of the concern when people talk about copyright and model weights.
It's a bit clouded by people who are just generally hoping that today's AI model weights are illegal for social reasons, but that's not the position I am trying to present. (I'm not really sure what we should do regarding societal impact.)
It's an interesting read and makes a good case for why none of the steps are directly copyright infringement, even if you can prompt the output to be (and in that case the person doing the prompting should be the one at fault, same as someone drawing something infringing directly).
It talks about copyright infringement in music, but it applies just as well to AI training, just substitute "scrambled file" with "model weights":
> The scrambled file still has the copyright Colour because it came from the copyrighted input file. It doesn't matter that it looks like, or maybe even is bit-for-bit identical with, some other file that you could get from a random number generator. It happens that you didn't get it from a random number generator. You got it from copyrighted material; it is copyrighted. The randomly-generated file, even if bit-for-bit identical, would have a different Colour. The Colour inherits through all scrambling and descrambling operations and you're distributing a copyrighted work,
Their business to take other people's suffering, and use that to write stories to sell views (and newspapers).
An egregious example is Kim Phuc https://en.wikipedia.org/wiki/Phan_Thi_Kim_Phuc
They featured a naked, burned girl on the front page of the New York Times to sell copies. I am not aware of any compensation the New York Times offered her. Her permission was never asked.
That picture has defined how the public has viewed her for the rest of her life.
They would argue that doing this is in furtherance of public goods such as people knowing what is going on and they could not realistically seek permission from every person much less compensate every person featured in a news photo or story.
I think that may be a valid argument. But doesn't AI also have the potential to benefit society. Isn't using news articles as input for AI making AI better and more useful to society. If the news can use people's images and suffering to benefit society (while making money), why can't their news stories be used to train AI to benefit even more people with knowledge?
Some of the pushback is the desire to collect fees or social capital for works, which is understandable, but at the same time that inherently creates walled gardens of information and the ability to combine (synthesize), something which could be considered outdated compared to what these algorithms could offer.
One somewhat reasonable outcome could be that for profit offerings must develop ways to limit excerpts and acknowledge sources, the latter similarly to how Google search works. I think it would be a terrible outcome if knowledge and AI abilities were splintered off to different product groupings (eg NYT only available via Apple AI).
But I don’t think it resolves all the problems. Newspapers are losing their business model. Maybe it is just creative destruction and we’ll have something better in their place. Maybe, but journalism is quite important in our societies.
I don't know if it's just not understood how these models work, or if it's just purposely misleading to try and cripple scary new tech.
It seems like writers trying to complain it's copyright infringement to have other writers read their works for inspiration.
But now, when the scenario is adjusted to be a network of ALPN sensors blanketing an entire metro area, am I just "purposely misleading to try and cripple scary new tech" when I object to such a system?
I understand that AI is capable of copyright infringement, I too can draw a batman symbol off the top of my head, and it seems the fix for this simply output filtering AI content. But look at any AI-copyright complaint and all they focus on is training.
Since you asked for someone to explain: IMO you're focusing on a mechanistic explanation of how ML happens, but if you take only a step back, the resulting models are really similar to a compressed blob of the training data. At times they regurgitate the training data almost exactly. This is what most people are seeing and (IMO, rightly) calling out as blatant infringement.
Slightly different topic: I think calling it "learning" is brilliant PR, but if you survey AI experts, they'll mostly agree it's not learning in the same way a human brain learns. You and I get more legal leeway than a computer, and it'll be interesting how that works out if/when AGI becomes a thing, but we're nowhere near there yet.
Ok, how much?
How would one determine all sources and their contribution weights for each produced completion from GPT?
(1) Make training opt-out trivial and fast
(2) if OpenAI still wants to use material in training, the can approach author with an offer.
Something like Google is a win-win proposition: Google gets useful results, while publisher gets clicks and ad impressions. But being included into AI training is not like that, authors get nothing for being included in the AI training set. So they should be free to charge whatever amount of money they want, or opt out completely.
Look, I publish (nearly) all my blog posts for _free_ on the web because I like doing that, and I _still_ don't want chatbots scraping my content to serve their algos (yes, I'm blocking them now via robots.txt) because the reason I put content out for free is partially because I want my name and my expertise out there. I'm part of a community of authors if you will. Call it selfish or self-serving if you will, but that's my right as the author.
Having stuff I've contributed to the internet get ingested into a model and regurgitated out such that my contributions are sidelined…well, that's just not acceptable, any more than a person taking my blog post, rewriting it with a handful of other brief sources, then republishing it under their name with no attribution back to me is acceptable.
Why would the buyer be responsible for pricing?
Once your content is online, it's subject to copy, piracy and manipulation of many forms, because it's on the WORLD WIDE WEB.
A US law can't enforce the world's network of computers and what they do with it's data.
Go ahead and sue OpenAI, I hope they lose.
I'm rooting for Open Source LLMs, that's where the real innovation is at, without censorship or copyright restrictions.
> By OpenAI's logic, any work you put online is fair game to be swiped and incorporated into the company's large language models.
With
> Any work you put online is fair game to be swiped and incorporated into a human's brain.
It's tough because the ability to copy an LLM is feasible, and an LLM is much more likely to be able to reproduce the underlying work with high fidelity, but there's no law that says that someone with a photographic memory can't look at your site data.