Ask HN: Isn't ChatGPT unfair to the sources it scraped data from?
> The model was trained using text databases from the internet. This included a whopping 570GB of data obtained from books, webtexts, Wikipedia, articles and other pieces of writing on the internet. To be even more exact, 300 billion words were fed into the system.
I believe it's unfair to these sources that ChatGPT drives away their clicks, and in turn the ad income that would come with them.
Scraping data seems fine in contexts where clicks aren't driven away from the very site the data was scraped from. But in ChatGPT's case, it seems really unfair to these sources and the work that the authors put, as people would no longer even to attempt to go to these sources.
Can this start breaking the ad-based model of the internet, where a lot of sites rely upon the ad income to run servers?
193 comments
[ 3.6 ms ] story [ 240 ms ] threadso in that case it wouldn't be unfair ... :-)
did the ChatGPT pay for the content it is using? that was the original question...
To me it looks more like memorizing enough of other employees' project contributions to try passing it all off as your own achievements in performance review.
I create a omniscient copyright detection bot and face it at everything you create 24 hours a day 7 days a week.
You go home and sing happy birthday to your kid. The bot gives you a non-monetary warning for using a copyrighted work without permission. No big deal, but it is on your permanent record.
It had been a stressful day so you take up your evening hobby of painting. You like nature scenes and trees and 30 minutes in you receive a violation, evidently Bob Ross has already done this and his surviving estate is now asking you to destroy the picture.
The next day you go to into your job at the corporate bureaucracy slinging lines of javascript. It's been a productive day so far and you have a few hundred new lines of code written and then the bots going off and HR and legal are ringing the phone within seconds. Turns out some comment you'd saw on Stack Overflow years ago was imprinted in your memory well enough you committed a copyright violation. Looks like you'll be losing your job.
Look at the videos flagged by youtube or copyright trolls, a lot of them are not actual copyright violations, but they are flagged anyway by the algorithm and removed or demonetized. And it takes a lot of work to fight those claims.
Me singing happy birthday at home or painting a picture for myself to relax is already demonetized. There doesn’t need to be an omniscient (and wrong) copyright bot to do that.
No, I didn’t.
I was hedging against the too-clever HN commenter coming back and saying, “The robot knew you were going to sell the painting” or “You sang the game on the Jumbotron at Yankee Stadium.”
(This is an active area of research, though, and version of GPT that could cite its sources is something people widely agree would be valuable.)
I can't seem to find anything on OpenAI's crawler agent, so I'm skeptical they're considering robots.txt at all.
I'd also argue that Google directing traffic to your website is a good alignment of incentives. ChatGPT spitting out answers derived from your work with nothing given back to you in return is not.
E.g. Summary of How to Win Friends and Influence People: Effective Steps to Better Interpersonal Relationships by Book Lyte
ChatGPT does more of a mashup with the learned data than humans need to, that'll do me.
In other areas of society where a bad thing cannot be stopped, we still use legislation to reduce the amount of it and mitigate some of the harm.
People get internet hostile at me for this question, but it really is that simple. They've automated you, and it's definitely going to be a problem, but if it's acceptable for your brain to do the same thing, you're going to have to find a different angle to attack it than "fairness".
With ChatGPT the traffic benefit isn’t there, so it feels like it isn’t a fair trade.
Google adding the context and data to their search results page also started blurring this trade making it unnecessary to click to the site the info was cleaned from.
Oh, wait, I'm not going to cite sources in a non-scientific work as this leads to madness. The following is a previous post of mine on HN
"Your mind exists in a state where it is constantly 'scraping' copyrighted work. Now, in general limitations of the human mind keep you from accurately reproducing that work, but if I were able to look at your output as an omniscient being it is likely I could slam you with violation after violation where you took stylization ideas off of copyrighted work.
RMS covers this rather well in 'The right to read'. Pretty much any model that puts hard ownership rules on ideas and styles leads to total ownership by a few large monied entities. It's much easier for Google to pay some artist for their data that goes into an AI model. Because the 'google ai' model is now more culturally complete than other models that cannot see this data Google entrenches a stronger monopoly in the market, hence generating more money in which to outright buy ideas to further monopolize the market."
I would say most knowledge about words/grammar/laws of nature can be taken for granted without a citation, but there are some important exceptions where things must be cited. I don't know how you'd reliably teach the difference to a computer though.
It's getting me to the point of refusing to use Google, or only use Google with "site:...". I mean, the site varies, but without site limits Google's becoming useless.
If ChatGPT were to be required to share its sources, they would need a completely different approach. I'm not commenting on whether or not that would be a bad thing, but it would render the current iteration completely useless. You can't strap a source-crediting mechanism on top of a transformers-based model after the fact.
I've read that ChatGPT is not connected to the net, but if it was: Couldn't you have it do a google search (or better yet corpus search) for the string it generated and then return the most significant matches (significance by string matching, not google rank)? It would be really crude, but wouldn't this just be a handful of lines of code that don't interfere with the "transformers-based model" code at all?
The other day I had GPT write a rap battle between Burger King and Ronald McDonald. One of the stanzas came back:
It turns out that yes, Ronald McDonald was first introduced in 1963. https://en.wikipedia.org/wiki/File:McDonald%27s_commercial_(... (from https://en.wikipedia.org/wiki/Willard_Scott#Created_Ronald_M... )So here's the challenge for you - who do you compensate for that line?
The complaint that people have isn't that GPT isn't citing its sources but rather that it isn't compensating the people who created the data that has that information.
... and now, if you're ever asked about historical clown trivia and pull out the "Ronald has been around since 1963", who should you give a royalty to? Me (for writing this), GPT (for making me aware of it), Wikipedia (for the source of my links in this post), the estate of Willard Scott for the Joy of Living (which Wikipedia cites), some random blog author that had some clown trivia on it that happened to have been part of the training set for GPT?
It isn't just monetary compensation that's important here.
I come at this from the point of view of a scientist who is expected to reference ideas. Not necessarily back to their original source, but at least back to a source that can theoretically point back to another link in the chain.
Sure, I can manually search for a reference based on what ChatGPT gave me. Or someone could spend a few minutes adding a few lines of code to ChatGPT to save millions of people some minutes of time.
-----
What would be awesome is an LLM that you can feed data to, and it can then write a paper based solely on the data you feed it.
I had it write a poem the other day in the style of Roses are read about coffee and bacon.
If this is something that someone considers to be a derivative work of other things... who do I credit? to which I got back: How do you credit that?---
> What would be awesome is an LLM that you can feed data to, and it can then write a paper based solely on the data you feed it.
https://platform.openai.com/docs/guides/fine-tuning
Based on a quick search the best credits would be ChatGPT as the arranger, and "Roud Folk Song Index number 19798" as the inspiration.
> "Joke: What did the astronomer say when the musician asked him to join his band? "I'm sorry, I don't do solos in the dark!""
> "How do you credit that?"
That you credit to ChatGPT. It's not referencing facts or discoveries, so credit isn't as important as it is for articles. If you want to credit an inspiration then I'm sure there's an index of joke forms out there that has an appropriate number to cite.
I can't actually find a definition for band in astronomy that is "a dark region in the sky with less stars." So it seems to be a pretty poor joke.
> https://platform.openai.com/docs/guides/fine-tuning
This does it solely based on the data you feed into it? And by data I mean scientific data that you discovered, and want formatted into a particular research article style.
Edit to add: Possible sources for the line "together they please me":
1) https://www.google.com/books/edition/Poetical_Works_of_Louis...
2) https://www.google.com/books/edition/Florio_s_First_fruites/...
The first is an exercise for the reader (and much better done and evaluated by the reader). The second is what people are concerned about.
> Why did you pick that index rather than some other source material?
I told you why references were important in scientific documents already.
If you are writing something that is synthesizing knowledge (not just reporting the facts), the "where are all the places were that knowledge came from" is an impossible task for human or machine.
If I ask GPT to create a poem in the style of Roses are Red about coffee and bacon - why should that request need to be citied to the same degree of scrutiny as an encyclopedia or research paper?
If, on the other hand, you're trying to use GPT to write such a paper... I would hold that you're doing it wrong. It doesn't do that well. The model is "about" transforming language. To do so, it has a fair bit of 'knowledge' that it contains to be able to do that accurately. OpenAI makes no claims about the accuracy of the content that GPT produces (its improved, it can more accurately answer data - but if you want to know the answer it is no better than your next door neighbor who has read a lot).
If you are claiming that the example of Bacon is Greasy poem that GPT wrote is infringing any more than a child's "roses are red, my cat is orange, his eyes are green, nothing rhymes with orange" then I believe you will face an uphill battle.
To say that there is plagiarism and infringement going on - it needs examples rather than a "I think it works this way and is just regurgitating material it was fed from elsewhere."
The difference is scale. At scale it becomes a problem.
Edit: I don't know how to satisfy all parties. This shakes the foundation of copyright. Perhaps we are all finding out how valuable good information truly is and especially in aggregate. We have created proto-gods.
This could be an excellent brain augmentation, trying to hamper it because we want to force people to drag themselves through underlying sources so those sources can try to steal their attention with ads for revenue is asinine.
And in the meantime, with ads no longer working, maybe crypto is actually useful for something here - lightning makes very small transactions possible with basically no fees, and makes it easy to programmatically pay for things. People hate being nickled and dimed, but a professional trying to construct an ML model could reasonably budget for use fees for fast unhindered access to quality training data. An agent could even evaluate its likelihood of learning something new/accurate vs the cost proposed by the server, and choose the subsets to pull.
Just a random idea, but I hope we don’t fight tooth and nail to preserve the trash heap of the internet’s current state.
If you previously interacted with people on this issue, you must know that.
It is fair for a single human to breathe, but not for a machine to use all oxygen on this planet at once, killing everyone else in the process.
"Learning is unfair" is not an argument you want to win.
> The scale and totality of the scraping is out of reach for a human.
Why?
Air is zero-sum. Knowledge is not.
If you became the first line, go-to source for the information of those websites, those websites would stop getting click-throughs. Eventually it would become less and less worthwhile (economically or emotionally) for the people keeping those sites running to keep them running. It would become more and more difficult for people to find those sites even if they are running, or even the archives of those sites.
So yes, eventually you'd stop people from reading them too.
Because it's false equivalence? ChatGPT isn't a human being. It's a product that is built upon data from other sources.
The question is if this data is legal to scrape, which it is: Web scraping is legal, US appeals court reaffirms [https://news.ycombinator.com/item?id=31075396].
As long as the content is not copyrighted and it's not regurgitating the exact same content, then it should be okay.
...it is? I didn't see that question raised in OP's text at all. What do legacy human legalities have to do with how AI will behave?
> Because it's false equivalence? ChatGPT isn't a human being.
Is this important? What is so special about human learning that it puts it in a morally distinct category from the learning that our successors will do?
It sounds like OP is concerned with the ad-driven model of income on the internet, and whether it requires breaking in order for AI to both thrive and be fair.
Well yes, it's the whole crux of the matter. Laws govern human behaviour. As of 2023, only living beings have agency. If I shoot someone with a gun, the criminal is me and not the gun. Being a deterministic piece of silicon, a computer is perfectly equivalent. Sure, it is important to start a discussion of potential nonhuman sentience in the future, but these AI models are not unlike any previous software in legal issues. It's bizarre to me how many people are missing this.
Very much this. I am too tired right now to engage with other responders, but thank you for articulating precisely the point I want to make.
Agreed. However, the previous 'legal issues' related to software and the emergence of the internet are also difficult to take seriously when considered on anything but extremely short time scales.
Every time we swirl around this topic, we arrive at the same stumbles which the legacy legal system refuses to address:
* If something happening on the internet is illegal, _where_ is it illegal? Different jurisdictions recognize different jurisdictional notions - they can't even agree on whose laws apply where. If you declare something to be illegal in your house, does that give it the force of law on the internet? Of course not. Yet, the internet doesn't recognize the US state any more than it does your household. It seamlessly routes around the "laws" of both.
* The "laws" that the internet is bound to follow are the fundamental forces of physics. There is no - and can be no - formal in-band way for software to be bound to the laws of men, because signals do not obey borders. The only way to enforce these "laws" are out-of-band violence.
* States continuously, and without exception, find themselves at a disadvantage when they make the futile effort to stem the evolution of the internet. For example, only 30 years ago (a tiny spec in evolutionary time scales), the US state gave non-trivial consideration to banning HTTPS.
I understand that people sometimes follow laws. But they also often don't. The internet has already formed robust immunity against human laws.
Whatever human laws are, they are not the crux of anything related to evolution of software. They are already routinely cast aside when necessary, and are very clearly headed for total irrelevance.
So not it's not a false equivalence.
> Web scraping is legal, US appeals court reaffirms
First, the case is not closed. [0]
Second, to draw an analogy, you can use scraping in the same way you can use a computer: for legal purposes. That is, you cannot use scraping to violate copyright, just as you cannot use a computer to violate copyright.
The following being my conjecture (IANAL), there is fair use and there is copyright violation, and scraping can be used for either—it does not automatically make you a criminal, but neither is it automatically OK. If what you do is demonstrably fair use presumably you’d be fine; but OpenAI with its products cannot prove fair use in principle (and arguably the use stops being fair already at the point where it compiles works with intent to profit).
[0] https://news.ycombinator.com/item?id=31079231
Fair use of course being the exception.
Now, as for accessing things like credentials that get left in unsecured AWS buckets is the bigger area where courts are less likely to recognize the legality of scraping. Never mind the fact that these people literally published their private data on a globally accessible platforms in a public fashion. I'm not a lawyer but I've seen reports of this leaning both directions in court, and yes, I've seen wget listed as a "hacker tool."
This is what happens when feelings matter more to the legal system than principles.
And before it's brought up, I may as well point out that no, I don't condone the actual USE of obviously private credentials found in an AWS bucket any more than I condone the use of a credit card that one may find on the sidewalk. Both are clearly in the public sphere, unprotected, but for both there is a pretty good expectation that someone put it there by accident, and that it's not YOUR credential to use.
Basically, getting back to the OP, ChatGPT hasn't done anything I've seen that'd constitute copyright infringement -- fair use seems to apply fairly well. As for the ad-supported model, adblockers did this all first. If you wanted to stop anything accessing your site that didn't view ads, there are solutions out there to achieve this. Don't be surprised when it chases away a good amount of traffic though -- you're likely serving up ad-supported content because it's not content you expected your users to pay for to begin with.
To be fair, so are you.
When a person "scrapes" a website by clicking through the link it registers as a hit on the website and, without filters being turned on, triggers the various ad impressions and other cookies. Also if the person needs that information again odds are they'll click on a bookmark or a search link and repeat the impression process all over again.
When an AI scrapes the web it does so once, and possibly in a manner designed to not trigger any ads or cookies (unless that's the purpose of the scrape). It's more equivalent to a person hitting up the website through an archive link.
That is, instead, one of the larger and vastly more important sociocultural issues that actually warrants attention, but never receives it in sufficient degree to address the problem, because, for example, we're arguing whether automated learning is "fair".
https://en.wikipedia.org/wiki/Luddite
Might as well ban computers since they automated and eliminated a lot of manual jobs.
The problem of humans with no money should be solved by a societ safety net and things like UBI.
Until things are, in fact, solved by whatever idea you might have, why should we just accept each new thing that makes our human lives more intolerable? How could you expect any rational person to have that kind of blind trust in a technology, much less "progress" itself, when every single aspect of our world shows that it is who owns the technology that actually benefits from it? I think it is much more crazy just totally rolling over for each new thing that takes your job than it is to maybe fight for your food and shelter.
I think we can do better than UBI, but either way, fighting against this unfairness is fighting for the things we need to continue with some shred of humanity, insofar as this technology is and will be an agent for the consolidation of labor and profit. Its all the same fight, and the historical luddites understood this consciously or not.
Who knows, maybe the internet would have been better off if some people were brave enough to smash some of Google's servers in like 2006..
I don’t do it on an industrial scale.
That is trivially disproved, as is the rest of your argument that follows from it as a premise.
There is really no reason to believe that what chatGPT or stable diffusion does is anything like what "your brain" does--except in the most superficial, inconsequential way.
Second, try applying this logic to literally anything else and you'll see why it's absurd:
"You can't ban cars from driving on sidewalks! If it's acceptable for people to walk on sidewalks, then it has to be acceptable for cars to drive on sidewalks, since it's just automated walking"
"You can't ban airplanes from landing in ponds. They fly 'just like' ducks fly! So if it's acceptable for them, it must be acceptable for airplanes too"
Why would it be incoherent to say “I’m okay with a person reading, synthesizing, and then utilizing this synthesis—but I’m not okay with a company profiting off of a computer doing the same thing.” What’s wrong with that?
But again, like you and others have said, it’s really not the same thing at all! All ChatGPT (or any other deep learning model) is capable of doing is synthesizing “in the most superficial way.” What a person does is completely different, much more interesting.
I also agree it's not the only argument and ultimate proof.
I don't at, this point, have an answer. I'm sure this miraculous new technology will survive the luddite attacks, but there will probably be some tense moments, and some jurisdictions will choose to be left behind.
Imagine if you were a webmaster and Google unilaterally decided to stop sending users to content you have worked to research and write, and instead aggregated it and showed the answer to user’s query entirely on its own pages, without any attribution or payment to you. Unimaginable, yet that is very much the scenario unfolding now. [1]
Scraping at this kind of scale is out of your (or any given individual’s) reach. It is, however, within reach of the likes of Microsoft (on whose billions OpenAI basically exists) and Google (who, to be fair, have not abused it in such a blatant way so far).
[0] It is clearly using someone else’s works for commercial purposes, including to create derivative works. (Again, it’s different from you creating a work derivative from someone else’s work you read previously, because in this case a corporation does it at scale for profit.)
[1] And the cynic in me says the only reason we are not yet out with pitchforks is simply because OpenAI is new and shiny and has “open” in its name (never mind the shadow of Microsoft looming all over it), while Google is an entrenched behemoth that we all had some degree of dissatisfaction with in the past and thus are constantly watching out for.
The difference is in scale.
A human video game designer can consume other' people's art, then sell their labor to a video game developer. The amount of value captured by the video game designer rounds down to zero in terms of percentage of economic value created by 'video game art'.
OpenAI can consume all of the video game artists, ever, create an art design product and capture a significant percentage of the economic productivity of video game art.
I don't think the word-by-word statistical mechanism of ChatGPT would stand up as a copyright defense in court. It's the output that counts, not the means of getting there. It'd be like me copying some copyright work word-for-word then trying to claim "well, your honor, I was only using that for inspiration, I was using my full creative abilities to write what I did, so you can't blame me if it's a word-for-word copy".
I think OpenAI (or any company with the resources to train such a model in the first place) could fairly easily self-police and check that what they are generating isn't an exact (or almost exact) copy of something it was trained on. It's a bit like the app Shazam/similar recognizing a song from a short snippet - you just need to generate some type of "hash code" for each generated sentence (or whatever level of granularity makes sense) and compare it to a database of "hash codes" from the source material it was trained on.
So, who gets credit for the word "mat" being generated in that context ? I guess any texts talking about cats and mats in close proximity may deserve some of the "credit", but it goes way deeper than that since why did ChatGPT choose to output such a trite sentence (albeit while only selecting one word at a time), rather that something else about cats or perhaps a more interesting thing that cats often sit in/on ...
People seem to assume that ChatGPT is pulling entire "facts" from various sources, but that's just not how it works - it's just feeding all the texts into a giant meat grinder of word statistics. It knows about words, not facts.
...yes?
OpenAI: "challenge incorrect assumptions"
1. You have a widely read spouse named Joe who reads constantly. He's got a good memory, and typically if you have a question you just ask him instead of searching for it yourself. Are you depriving Joe's sources of your eyeballs?
2. Many books summarize and restate other books. If I read Cliff's Notes on a book, for example, I can learn a lot about the original book without buying it. Is this depriving the author?
3. I have a website that proxies requests to other websites and summarizes them while stripping out ads.
So which of these examples are a better metaphor for what a LLM does?
I don't know. The fact is, LLMs are a new thing in our tech and culture and they don't quite fit into any of our existing cultural intuitions or norms. Of course it's ambiguous! But it's also exciting.
"If you applied the same set of rules to a human, how exactly would that look"
Simply put culture is the copying of each others ideas. When one of us started banging rocks together to make them sharp they didn't sell this idea to others, at best they traded sharpened arrows for something else.
The big issue with humans is we are commonly very conservative in our ideas. "Yesterday I did X, today I did X, and tomorrow I'll do X", fine and dandy until tomorrow a machine does X for nearly free. Instead of figuring out how to adapt our economic systems to deal with new systems of cheap and plenty the fearful and the greedy are looking for ways to maximize the amount they can profit or hold it back to maintain status quo.
A typical single-spaced page is 500 words long
That’s 179,280,000 full pages of text.
I wonder if they excluded any duplicated text.
I’ve only done image classifiers and object detectors so I was assuming they must be trained with similar pure datasets.
The neural net model is condensed to 800 GB.
https://www.springboard.com/blog/data-science/machine-learni...
Note that the "compression" there also includes the "intelligence" that it presents - you might be able to get some powerful compression of English text... but you can't ask a gzip file to come up with a joke about cats and dinosaurs.
We can only hope. It’s unfair to someone that my browser can ask your server for a page, I see an ad for random bullshit nobody would ever care about, and money changes hands behind the scenes and that counts as an economic transaction which boosts GDP. It’s unfair (in my favour) that I can piggy back off this to get things for free.
And when I say “someone“ I suspect “everyone”. Sadly spending money advertising “Yorkshire woman finds guaranteed way to win on the horses” doesn’t seem to have caused anyone to run out of money and have the whole thing collapse yet. And it’s unfair on real small businesses with products paying for adverts which people don’t see or are clicked by bots or are misreported and all they can do is throw money at Google and Facebook and hope.
If you are using GPT as a research tool as opposed to asking your friend who is an expert int the subject, are you citing your friend when you write the paper - or are you going back and finding sources that then back your friend's point up?
It maybe was unfair to telephone operators when connection automation was implemented, as it made operators obsolete, but the older model couldn't scale, the same way reading text from source doesn't scale for human productivity.
Also, I agree exactly. Advertising is increasingly useless. It's a tax on knowledge and it's gross. I can't wait for it to die.
I want to only pay for the stuff I use.
I'm actually not really sure I have an opinion on the ethics of it. Same argument as Adblock. You don't get to control how people consume your content if you put it out in the world for free. That goes for profiles, or articles, reddit posts, StackOverflow, etc. The only thing that's ironic is that large tech companies throw a fit whenever you want to turn the tables and scrape them.
> What's the New York Times scrambled egg recipe?
GPT returns the exact recipe. If I were NYT I'd be frustrated. Their content is now showing without the ad views or paywall.
On the other hand, the focus on the potential of ChatGPT's natural language processing capabilities highlights the significance of learning and using LLM (Language Models) in data handling. The utilization of LLM can potentially lead to a future where traditional databases become obsolete and are replaced by advanced language models. As such, the development and integration of LLM in our daily lives and processes can bring about many benefits and possibilities.