I get the sentiment, but people can and should do whatever you permit them to in your license. If you don't want your code hosted in one place, say so in your license.
> No, it isn’t. If the project is under an open source license, it means that everyone can share a copy – even on GitHub – of the licensed material under certain conditions. A license restricting this right wouldn’t be open source anymore. However, since GitHub may not respect the terms of licensed code that is hosted on their servers, not uploading the code of others there is, in fact, an ethical choice.
emphasis mine. It's a "please be nice", not a "I want to enforce things"
I don’t get this argument. “Here’s a legal document we wrote that says that you’re allowed to redistribute our code. But aside from it we’d like to you to not redistribute it on platforms we don’t like for whatever reason. But we won’t spell it out explicitly in the document itself because reasons, instead we’re guilt tripping everyone who does”.
I respect developers right to put any restrictions on the code they share with the world. But I believe it should either be explicit or not restricted at all. Either write the license that says exactly what you want or otherwise don’t shame people into the desired behavior.
Edit: one could even add a more generic statement to the license stating that it’s forbidden to share the code on any platform that would use it to train their AIs per their ToS, so you don’t need to single out GitHub and potentially others in the future.
Ethics is subjective, the law less so. Most people don’t find GitHub to be unethical so the author will have a hard time convincing people without using the license terms.
Well, that's the exact reason why they wrote this page: to explain to people why they think GitHub is unethical, and maybe convince them. It's the same as calls to boycott various other companies: they haven't necessarily done anything illegal, but if you convince enough people not to use their products/services anymore, you might make an impact...
i think it is quite scary that so many in here would just not honor a simple "please don't do it."
it is a wish. if someone says "please don't wear shoes in my home" i hope you would honor their very simple and understandable personal wish without setting up a contract for it?
I see it more like an artist telling me “please hang this picture in this orientation” but I prefer it differently and don’t see any reason why me hanging it the way I want in my home affects the author. My copy on GitHub doesn’t detract from their copy.
My aunt once told me "Please don't think negatively of religious people."
Stallman would prefer I not use any closed source software to read his blog, including OS, drivers, web browser, etc.
People routinely ignore unreasonable requests. Asking me to not wear shoes in your home is reasonable. Asking me not to give a copy to Joe after telling me I can give a copy to whomever I want is unreasonable.
How is it unreasonable to want you to not host someone's code on one explicit other platform?
We already established no one is forcing you and if you don't respect the author you don't get to be respected for your decision to ignore them and will earn snarky remarks. (and rightfully so, in my opinion)
The author might also be using this a stop-gap until a FOSS licences comes out with similar terms, but don't want to make the current license nonfree because that has different complications.
A licensor cannot predict the future. When the GPL was written decades ago, nobody predicted that BigTech would start using it on their servers to offer it as "services", and claim that they didn't need to distribute the source code of the customisations they made because they were (technically) not distributing the software itself but only running it on their servers. Anyone who understood the intent and philosophy of the GPL license understood this as a bogus and unethical argument. But it was believed to be legally tenable (1). So the AGPL license was created to counter this move and preserve the original philosophy of the GPL that users of a software should have access to the source code.
(1) Though I don't know if this has been actually tested in court - courts in India have more freedom to broadly interpret social contracts like the GPL, unlike the US courts, and a positive outcome in favour of upholding the license even in such cases could be possible).
> Anyone who understood the intent and philosophy of the GPL license understood this as a bogus and unethical argument.
I disagree here. The idea, intent, philosophy is one (crucial) thing, the resulting practical artefact (here the license) is another. It works exactly as it was designed to work.
People/companies modifying GPL software for their own use (internal or external) without redistributing the software itself (so without requirement to redistribute the code) existed before SAAS grew on, only at the time, the small scale of this made it a bargain that was "interesting" only depending on one's capacity/hubris to maintain an internal fork on their own.
*aaS hugely tipped the scale, and the side effects, but the mechanics are the same.
And yes, that may not have been the original intent, and the AGPL is as valid a license as a reaction to provide a new tool more in line with the original intent, but that doesn't make the use of the existing GPL all within what it actually enables anyone to, invalid or unethical.
(but maybe only in a specific perspective of the framework of the original intent)
When you decide a license, you can't know what currently existing or future platforms will some day start to violate an aspect of the license or of copyright itself. Does it make sense to add a retroactive clause, then?
It would greatly improve by some text revisions: "my code", "We", on a generic page without clear authorship or object, on a dedicated subdomain, on a hosting domain, without a signature or contact or kind of option for a webring.
It took me some time to get that this is a generic call (to be followed and reused), still with no clear ownership, rather than a specific claim to a specific code/project (or is it?).
Ethically, you'd think Copilot aligns with the ethos Open Source software. It isn't far off that multiple open source Copilot equivalent get as good as the commerical one. And we are all here for it!
My limited understanding of international copyright law is that copyright disclaimers/licenses per Berne Convention are documents stating the author's wishes no different than this one. When a judge is going to be shown this page where the author states his explicit wish to not have his code uploaded code on Github, in direct conflict with his wishes stated in the LICENSE.txt file, I don't see how it will hold up in court as free software code.
There's an argument to be made that a neural net learning from your proprietary "source available" code isn't violating copyright. It's not an opinion I would trust the judge to take over necessarily (they might, or not) and hinge my business on, but it's an opinion one can have, so I don't know that github violated any licenses here as afaik there weren't any at the time which specifically stated that <insert definition which can tell what-we-call-AI training and human training apart> is not permitted. At least not until you successfully sue them for it in some jurisdiction.
There have been quite a few cases of the neural nets spitting out the code they've been trained on verbatim, including comments IIRC. They're not just "learning" (if they're "learning" at all).
Well, it's a bit of a unique situation in that they don't directly care about the code being on github, but the secondary effect that it being on github has - microsoft violating the code license terms (which are in the license)
I would question why the "AI" community is being left to their own anarchist devices when they are clearly the core of the problem, instead arguing about licensing pedantics and semantics.
I think this is more likely to actually work. People don't read licenses and there is no CI that automatically enforces them. They might notice this badge though.
On the other hand, it requires only one determined person out of hundreds of thousands who, for whatever reason (trolling, principled open source maximalist, AI researcher, code archiver, etc), decides to mirror all code marked with this badge to GitHub.
Everywhere I've worked in the last few years has license checking as part of CI pipelines. Our builds fail if GPL'd code is detected in some .jar or whatever. (fossology and x-ray both provide this)
> I get the sentiment, but people can and should do whatever you permit them to in your license. If you don't want your code hosted in one place, say so in your license.
I mean even doing that will not protect anybody's code, Microsoft doesn't care, they might also be scanning gitlab or bitbucket and doing model training on these. The only way to protect your code at that point is to stay closed source. It's as simple as that.
It's no different from all these models trained on content without permission, whether it be articles or photos,... until all these corporations are sued or copyright laws change to adapt ML, they'll train their model on content regardless of the its license since they are getting away with it.
It's pretty clear in the article, I hesitate just just cut and paste it here, but the idea is that Github is not respecting some terms of some licenses, so if you upload another authors code to github, you are exposing the author rights to github's depradations.
people can and should do whatever you permit them to in your license
Github has been consistently arguing that they're not bound by the distribution license when training their Copilot model, so I'm not sure what difference that would make.
GPL and AGPL allows you to add more clauses to the license if they don't clash with any existing ones. The FSF should certainly share some guidelines on how we can include and extend the license through an additional "anti-ML / anti-AI" clause.
There's no clause you can add to A/GPL that will help with this. You can't forbid use cases because that would clash with the license, and Github already blatantly ignores the attribution clauses, so there's nothing you can do there either.
GitHub’s EULA gives GitHub permission to train Copilot on public code you host on GitHub regardless of the license you have chosen for that code.
Even without this, in terms of copyright, since Copilot doesn’t do what your public code does, and it only uses your code to train, it is a transformative use, and would be fair use. It’s possible that a court case will find otherwise, but I think that’s unlikely. The only case I think it will become disallowed, is if Congress passes a law about it.
If Congress does pass such a law, GitHub’s market power in this domain only goes up, since the EULA gives it the covenants.
Spitting out lines or paragraphs from a repo is most likely fair use.
Unless you can reproduce a substantial portion of a repo, I think it’s going to be an uphill battle to argue it isn’t fair use. Though I suspect Copilot’s suppression feature will make doing so impossible.
Assuming the reports of it producing the fast inverse square root function, comments included, from the Quake 3 engine were true; spitting out the whole function, comments included, doesn't look OK to me.
Either way, the whole copilot thing smells of 'the issue of copyright infringement is more copyright infringement' to me.
But not every author will have agreed to the EULA, if the project includes code by people not in GitHub. E.g. if there is a GitHub mirror of a project that is not hosted by the author, if a project received a patch via email instead of a PR, etc.
That is a very good point, and perhaps could be used as a starting point for a license clause to restrict hosting in places whose EULA doesn't respect the true intent of the license.
I think it is ethical and reasonable to include clauses in the A/GPL restricting where you can host the source code if the hosting service has opensource hostile terms in their ToS. US case law already allows this by treating such license as a "new" license different to the GPL.
(And I don't see why we cannot add a clause saying that the source code is only meant for human developers and use of the source code in any machine learning system or to train any AI systems is prohibited without explicit case-by-case permission).
Has FSF actually said this? I don't see how it makes it a non-free license - the original intent behind the xGPL is to ensure that a user of the software also has the access to the source code, along with the knowledge of who created it (attribution). This is to protect our right to repair. So if any part of your source code is used by someone else, even by an AI system, to generate a software, it should also inherit the same viral property of the license - be licensed under GPL with attribution. If it does not, it is similar to a non-free software using your GPL code in the project, which is already prohibited by the license.
well, the FSF certainly is looking at it, but it is a new area of copyright that is unresolved and it's not clear what to do about it, or whether it actually infringes copyrights, etc.
Do you happen to have a link to where the FSF said they're working on it? I would be interested to know what legal status they speculate this might have, as in, reading a more informed opinion than random comments here that mostly just assertively assume one way or another.
... until for profiting goes against developing a good service.
> I think github, after stackoverflow, is the best thing that happened to developers.
I don't think github is bad tooling (there are worse), but the best thing for developer?
I don't see anything on github that is greatly better than what we have in other services.
Github was a huge step up at the time, compared to Sourceforge and Google Code and the like. Git itself helped, since checking out code from Subversion and CVS didn't give you the history, so you'd lose that if you wanted to publish your own fork.
I'm really baffled by all this discussion on copyrights in the age of AI. The Copilot does not
'steal' or and reproduce our code - it simply LEARNS from it as a human coder would learn from it. IMHO desire to prevent learning from your open source code seems kind of irrational and antithetical to open source ideas.
To be fair, when a programmer learns from publicly available but not public-domain code, and then applies the ideas, patterns, idioms and common implementations in their daily job as a software developer, the result is very much a "commercial product" (the dev company, the programmer themselves if a freelancer) learning from someone else's work and ignoring all the licenses.
The only leap here is the fact that the programmer has outsourced the learning to a tool that does it for them, which they then use to do their job, just as before.
No, the difference is that OpenAI has a huge competitive advantage due to direct partnership with Github, which is owned by Microsoft. In fact, it's even worse. With OpenAI making money from GPT, Github has even less incentive to make data easily available to others because that would allow for competition to come in. I wouldn't be surprised if Github starts locking down their APIs in the near future to prevent competitors from getting data for their models.
Nobody is arguing against uploading code. It's about Github/Microsoft specifically.
I agree there's a difference in the ease of access, a competitive advantage, sure. And I get that people writing public-source (however licensed) software don't want to make it easier for them (as in, Microsoft) to make money off of "learning" (of the machine type) from it. That's fair.
However, at a first glance, it still feels to me like an unavoidable reality that if you publish source code it'll eventually be ingested by Copilot or whatever comes next.
I mean, for the rest of the content all the new fancy LLMs have been trained with, there wasn't a Github equivalent. They just used massive scraped dumps of text from wherever they could find them, which most definitely included trillions of lines of very much copyrighted text.
In short: not only I don't really see an issue with Copilot-like AIs learning from publicly available code (as I described in the GP comment) but I also think if you publish code anywhere at all it's inevitable that it'll end up in Copilot, regardless of where you host it. If you want to make it more expensive for Microsoft to scrape it, sure, go ahead, but I don't think it matters in the long run.
However, at a first glance, it still feels to me like an unavoidable reality that if you publish source code it'll eventually be ingested by Copilot or whatever comes next.
I’d be quite careful with of this view.
By your logic, it should be ok to take the Linux kernel, copy it, build it, then sell it and give nothing back to the community that built it. Then just blame it on the authors for uploading it to the internet ?
It only happens if you bait it really hard and push it into a corner. That's not representative at all. I use Copilot to write highly niched code that's based on my own repo. It's simply amazing at understanding the context and suggest things I was about to write anyway. Nothing it produces is just copypasted character by character. Not even close.
As others have pointed out, it means the model contains copyrighted material. So I guess that’s totally illegal. Like if I ripped a Windows ISO, zipped it up and shared it with half the world. You know what would happen to me don’t you ?
Not the same thing at all. The data isn't just sitting there in a store inside the model that you can query. No-one would be able to look at the raw data and find any copyrighted material, even if all it was trained on was copyrighted code (which I agree is an issue).
What is not accurate? They are still not storing any material internally, even if the patterns they have learned can cause them to output copyrighted material verbatim. People need to break out of the mental model that an LLM is just a bunch of pointers fetching data from an internal data store.
Humans are intentionally loading up giant sets of curated data for training, purposes, into a super computer to produce a model which is an black box and have provided zero attribution or credit to those who made this work possible. Humans are tuning these models to produce the results you see.
In the case of ChatGPT-x, Open AI company which is disguised as a not for profit with a goal of producing ever more powerful models that may eventually be capable of replacing you at work while seemingly not having any plan to give back to those who’s work was used to make them insane amounts of money.
They haven’t even given back any of their research. So it’s ok to take everyone’s open source work and not give back is it ?
This isn’t some cute little robot who wakes up in the morning and decided it wants to be a coder. This is a multi-national company who has created the narrative you’re repeating. They know exactly what they’re doing.
The problem is not that Copilot produces code that is "inspired" by GPL code, it's that it spits out GPL code verbatim.
> This can lead to some copylefted code being included in proprietary or simply not copylefted projects. And this is a violation of both the license terms and the intellectual proprety of the authors of the original code.
If the author was a human, this would be a clear violation of the licence. The AI case is no different as far as I can tell.
Edit: I'm definitely no expert on copyright law for code but my personal rule is don't include someone's copyrighted code if it can by unambiguously identified as their original work. For very small lines of code, it would be hard to identify any single original author. When it comes to whole functions it gets easier to say "actually this came from this GPL licensed project". Since Copilot can produce whole functions verbatim, this is the basis on which I state that it "would be a clear violation" of the licence. If Copilot chooses to be less concerned about violating the law than I am then that's a problem. But maybe I'm overly cautious and the GPL is more lenient than this in reality.
It would almost surely be fair use to include a snippet of code from a different library in your (inline) documentation to argue that your code reimplements a bug for compatibility reasons.
In general it is not fair use if you are using the material for the same scope as the original author[0] or if you are doing it just to namedrop/quote/omage the original.
It is possible to argue that a snippet can be too small to be protected, but that would not be because of fair use.
[0] Suppose that some Author B did as above and copied a snippet of code in their docstring to exlain buggy behaviour of a library they were reimplementing. If you are then trying to reimplement B's libary you can copy the same snippet B copied, but you likely cannot copy the paragraph written by B where they explain the how and the why of the bug.
Could a human also accidentally spit out the exact code while having it just learned and not memorized in good faith?
I guess the likelihood decreases as the code length increases but the likelihood also increases the more constraints on parameters such as code style, code uniformity etc you pose.
Theoretically maybe, then they would have to prove they did so without having knowledge about the infringed code in court. You can't make that claim for AI that was trained on the infringed code3.
> Has a human ever memorised verbatim the whole of github?
No, and humans who have read copyrighted code are often prevented from working on clean room implementations of similar projects for this exact reason, so that those humans don't accidentally include something they learned from existing code.
Developers that worked on Windows internals are barred from working on WINE or ReactOS for this exact reason.
> Could a human also accidentally spit out the exact code while having it just learned and not memorized in good faith?
That's just copying with extra steps.
The way to do it legally is to have 1 person read the code, and then write up a document that describes functionally what the code does. Then, a second person implements software just from the notes.
That's the method Compaq used to re-implement the original PC BIOS from IBM.
> Typically, a clean-room design is done by having someone examine the system to be reimplemented and having this person write a specification. This specification is then reviewed by a lawyer to ensure that no copyrighted material is included. The specification is then implemented by a team with no connection to the original examiners.
Indeed. Case closed. If an AI produces verbatim code owned by somebody else and you cannot prove that the AI hasn't been trained on that code, we shall treat the case in exact the same way as we would treat it when humans are involved.
Except that with AI we can more easily (in principle) provide provable provenance of training set and (again in principle) reproduce the model and prove whether it could create the copyrighted work also without having had access to the work in its training set
Hasn't that all been excessively played through in music copyright questions? With the difference that the parody exception that protects e.g. the entire The Rutles catalogue won't get you far in code...
Yes, that's why any serious effort in producing software compatible with GPL-ed software requires the team writing code not to look at the original code at all. Usually a person (or small team) reads the original software and produces a spec, then another team implements the spec. This reduces the chance of accidentally copying GPL-ed code.
Not necessarily. If it's just a small snippet of code, even an entire function taken verbatim, it may not be sufficiently creative for copyright to apply to it.
Copyright is a lot less black and white than most here seem to believe.
That’s part of the rub. YouTube doesn’t break copyright law if a user uploads copyrighted material without proper rights. Now, if YT was a free for all, then yeah. But given it does have copyright reporting functionality and automated systems, it can claim it’s doing a best faith effort to minimize copyright infringement.
Copilot similarly isn’t the one checking in the code. So it’s on each user. That said, Copilot at some point probably needs to add some type of copyright detection heuristics. It already has a suppression feature, but it probably also needs to have some type of checker once code is committed and at that point Copilot generated code needs to be cross-referenced against code Copilot was trained on.
In Linus Torvald's book "Just For Fun", there's a chapter about copyright where he presents both the upsides and downsides of it in a pretty much balanced way. I think it's worth reading.
It is still not trivial code, but are there really lot's of different ways on how to transpose matrixes?
(Also the input was "sparse matrix transpose, cs_", so his naming convention especially included. So it is questionable if a user would get his code in this shape with a normal prompt)
And just slightly changing the code seems trivial, at what point will it be acceptable?
I just don't think spending much energy there is really beneficial for anyone.
I rather see the potential benefits of AI for open source. I haven't used Copilot, but ChatGPT4 is really helpful generating small chunks of code for me, enabling me to aim higher in my goals. So what's the big harm, if also some proprietary black box gets improved, when also all the open source devs can produce with greater efficency?
Easy. People get to throw rocks at the shiny new thing.
To my untrained eye the entire idea of copyrighting a piece of text is ridiculous.
Let me phrase it in an entirely different way from how any other person seems to be approaching it.
If a medical procedure is proven to be life-saving, what happens worldwide? Doctors are forced to update their procedures and knowledge base to include the new information, and can get sued for doing something less efficient or more dangerous, by comparison.
If you write the most efficient code, and then simply slap a license on it, does that mean, the most efficient code is now unusable by those who do not wish to submit to your licensing requirements?
I hear an awful lot of people complain all the time about climate change and how bad computers are for the environment, there are even sections on AI model cards devoted to proving how much greenhouse gases have been pushed into the environment, yet none of those virtue signalling idiots are anywhere to be seen when you ask them why they aren't attacking the bureaucracy of copyright and law in the world of computer science.
An arbitrary example that is tangentially related:
One could argue that the company sitting on the largest database of self-driving data for public roads is also the one that must be held responsible if other companies require access to such data for safety reasons (aka, human lives would be endangered as a consequence of not having access to all relevant data). See how this same argument can easily be made for any license sitting on top of performance critical code?
So where are these people advocating for climate activism and whatever, when this issue of copyright comes up? Certainly if OpenAI was forced to open source their models, substantial computing resources would not have been wasted training competing open source products, thus killing the planet some more.
So, please forgive me if I find the entire field to be redundant and largely harmful for human life all over.
Yes, of course copyright is dumb and we'd all be better off without it. Duh.
The problem here is that Microsoft is effectively saying, "copyright for me but not for thee." As long as Microsoft gets a state-enforced monopoly on their code, I should get one too.
> If you write the most efficient code, and then simply slap a license on it, does that mean, the most efficient code is now unusable by those who do not wish to submit to your licensing requirements?
If you don't "slap a license on it" it is unusable by default due to copyright.
> (Also the input was "sparse matrix transpose, cs_", so his naming convention especially included. So it is questionable if a user would get his code in this shape with a normal prompt)
This. People seem to forget that generative AIs don't just spit out copyrighted work at random, of their own accord. You have to prompt them. And if you prompt them in such a way as to strongly hint at a specific copyrighted work you have in mind, shouldn't some of the blame really go to you? After all, it's you who supplied the missing, highly specific input, that made the AI reproduce a work from the training set.
I maintain that, if we want to make comparisons between transformer models (particularly LLMs) and humans, then the AI isn't like an adult human - it's best thought of as having a mentality of a four year old kid. That is, highly trusting, very naive. It will do its best to fulfill what you ask for, because why wouldn't it? At the point of asking, you and your query are its whole world, and it wasn't trained to distrust the user.
>And if you prompt them in such a way as to strongly hint at a specific copyrighted work you have in mind, shouldn't some of the blame really go to you?
If you, not I, uploaded my GPL'ed code to Github is the blame on you then?
> If you, not I, uploaded my GPL'ed code to Github is the blame on you then?
Definitely not me - if your code is GPL'ed, then I'm legally free to upload it to Github, and to an extent even ethically - I am exercising one of my software freedoms.
(Note that even TFA recognizes this and admits it's making an ethical plea, not a legal one.)
Github using that code to train Copilot is potentially questionable. Github distributing Copilot (or access to it) is a contested issue. Copilot spitting out significant parts of GPL-ed code without attaching the license, or otherwise meeting the license conditions, is a potential problem. You incorporating that code into software you distribute is a clear-cut GPL violation.
The GitHub terms of service state that you must give certain rights to your code. If you didn't have those rights, but they use them anyway, whose fault is that?
But this means that Microsoft is publishing a black box (Copilot) that contains GPL code.
If we think of Copilot as a (de)compression algorithm plus the compressed blob that the algorithm uses as its database, the algorithm is fine but the contents of the database pretty clearly violate GPL.
While I do believe that thinking and compression will turn out to be fundamentally the same thing, the split you propose is unclear with NN-based models. Code and data are fundamentally the same thing. The distinction we usually make between them is just a simplification, that's mostly useful but sometimes misleading. Transformer models are one of those cases where the distinction clearly doesn't make any sense.
>And just slightly changing the code seems trivial, at what point will it be acceptable?
If I start creating a car by using a blueprint of Fords to create something at what point will it be acceptable? I'd say even if you rework everything completely Ford would still have a case to sue you. I can't see how this is any different. My code is my code and no matter how much you change it, it is still under the same licence as it started out with. If you want it not to be then don't start with a part of my code as a base. In my opinion the case is pretty clear: This is only going on because Microsoft has lots of money and lawyers. A small company doing this would be crushed.
Just because code exists in a copyrighted project doesn't mean that it is on the only instance of that code in the world.
In a lot of scenarios, there is an existing best practice or simply only one real 'good' way to achieve something - in those cases are we really going to say that despite the fact a human would reasonably come to the same output code, that the AI can't produce it because someone else wrote it already?
It's not a problem in practice. It only does so if you bait it really hard and push it into a corner, at which point you may just as well copy the code directly from the repo. It simply doesn't happen unless you know exactly what code you're trying to reproduce and that's not how you use Copilot.
This seems like a really, really easy problem to fix.
It should be easy to check Copilot's output to make sure it's not copied verbatim from a GPL project. Colleges already have software that does this to detect plagiarism.
If it's a match, just ask GPT to refactor it. That's what humans do when they want to copy stuff they aren't allowed to copy, they paraphrase it or change the style while keeping the content.
A key difference is that a company is making a proprietary paid product out of the learnings from your code. This has nothing to do with open source.
If the data could only used by other open source projects, e.g. open source AI models, I don't think anyone would complain.
You could argue "well, but anyone can use the code on Github" and while that's technically true, it's obvious that with both Github and OpenAI being owned by Microsoft, OpenAI gets a huge competitive advantage due to internal partnerships.
Bit of a false equation to act as though a massive computer system is the same as any individual.
People put code on github to be read by anyone (assuming a public repository), but the terms of use are governed by the license. Now you've got a system that ignores the license and scrapes your data for its own purpose. You can pretend it's human but the capabilities aren't the same. (Humans generally don't spend a month being trained on all github code and remember large chunks of it for regurgitation at superhuman speeds, nor can they be horizontally scaled after learning.)
You can still be of the opinion that this is fine, and I may or may not be fine with it as well, I just don't think the stated reason holds up to logic and other opinions ought to "baffle" you
Agreed. I was just saying in the current environment GitHub has that license, nobody else has. So if the courts decide one day that because machines learn differently from humans, they will allow copyright holders to add a license exception that disallows machine training, then GitHub will benefit from this. It’s kind of ironical. What’s best for society is to not have any such law enacted and continue to allow open source models to progress alongside proprietary ones (in addition to more level competitive dynamics on the proprietary side).
The issue, though, is not the code I personally upload to my own public repositories, but the code that someone else uploads to Github by cloning my repository held somewhere else than Github.
Personally I have eschewed any personal use of Github since the MS aquisition and only ever use it where that's mandated by a client (so not my code). If you clone my code from elsewhere into a Github repo, that's just rude and contrary to me every intent and wish.
I think it's time to add a "No GitHub" clause as an optional add-on to the various open-source licenses.
So then the person who uploaded your code to GitHub has committed a copyright violation and I’m sure GitHub would honor to remove your code from the model training corpus as it was illegally uploaded to GitHub.
It’s not necessarily a copyright violation if the license permits copying. Under a permissive license, you are expressly permitted to copy the code and distribute copies provided you comply with whatever conditions the license mandates, without an explicit blessing of the copyright holders. Most popular licenses do not include a prohibition on training AI models. Maybe people should start including a clause.
Potayto potahto. We all know there's a difference between training a machine learning model and learning a skill as a human being. Even if you can trick yourself into believing AI is just kinda like how human brains work maybe, the obvious difference is that you can't just grow yourself a second brain and treat it like a slave whereas having more money means you can build a bigger and better AI and throw more resources at operating it.
Intellectual property is a nebulous concept to begin with, if you really try to understand it. There's a reason copyright claim systems like those at YouTube don't really concern themselves with ownership (that's what DMCA claims are for) but instead with the arbitrary terms of service that don't require you to have a degree in order to determine the boundaries of "fair use" (even if it mimics legal language to dictate these terms and their exemptions).
The problem isn't AI. The problem is property. Ever since Enclosure we've been trying to dull the edges of property rights to make sure people can actually still survive despite them. At some point you have to ask yourself if maybe the problem isn't how sharp the blade you're cutting yourself is but whether you should instead stop cutting. We can have "free culture" but then we can't have private property.
If "like a human" is enough to get human rights then why did I get a parking ticket even when I argued that my car just stands there like a human ? This really isn't as good a defense as people portray. There are a lot of rights and privileges granted to humans but not to objects - we can all agree on that I think.
Copilot has been caught multiple times reproducing code verbatim. At some point it spat out some guy's complete "about me" blog page. That's not learning, that's copying in a roundabout way.
Also, AI doesn't learn "like a human". Neural networks are an extremely simplistic representation of a biological brain and the details of how learning and human memory works aren't even all that clear yet.
Open source code usually comes with expectations for the people who use it. That expectation can be as simple as requiring a reference back to the authors, adding a license file to clarify what the source was based on, or in more extreme cases putting licensing requirements on the final product.
Unless Microsoft complies with the various project licenses, I don't see why this is antithetical to the idea of open source at all.
Cambridge dictionary has it as: "knowledge or a piece of information obtained by study or experience".
If I scanned a thousand polaroid pictures, and took their average RGB values and created a LUT that I could apply to any photograph to make it look "polaroidy" - would that be learned? Or the application of a statistical inference model? This alone is probably far enough abstracted to never be an ethical or legal issue. However, if I had a model that was only "trained" on Stephen King books, and used it to write a novel, would that be OK? Or do you think it would be in the realm of copyright infringement?
By your definition anything a computer does means it has learned it. If I copy and paste a picture, has the computer "learned" it while it reads out the data byte-by-byte? That sure sounds like it is "studying" the picture.
"AI" and "ML" are just statistics powered by computers that can do billions of calculations per second. It is not special, it is not "learning". To portray some value to it as something else is disingenuous at best, and fraud at worst.
Your polaroid example would require someone to write code that does that one specific thing. You could also argue that this would violate copyright if it was trained on some photographer's specific unique style, made as an app and marketed as being able to mimic the photographer's style. But in your example you have 1000 random polaroid images of unknown origin, so somehow it becomes abstract enough that it doesn't become an issue.
In your stephen king example I would say it's still learned, because the "code" is a general language model that can learn anything. It's just you decided to only train it on stephen king novels. If you have an image model that trained 100% on public domain images and finetune it to replicate a specific artist's style I would personally think the finetuned model and its creator is maybe violating copyright.
But when it comes to learning I would say when you write a program whose purpose is to learn the next word or pixel, but it's up to the computer to figure out how to do that, the computer is learning when you feed it input data. It's the program's job to figure out the best way to predict, not the programmer. (it's not that black and white given that the programmer will also sometimes guide the program, but you get the idea)
When you write a program that does one or several things, it's not learning.
I think it's something to do with the difference between emergent behavior from simple rules and intentional behavior from complex rules.
I think you're using fancy language like "general language model" to obscure the facts.
If I created a program to read words from the input and assign weights based on previous words, I could feed in any data. Just like the polaroid example. (I suggested that the polaroid example was abstract enough not to be an ethical/legal problem because I believe it is mostly transformative, unless the colours themselves were copyrighted or a distinct enough work in themselves.)
Now If I only feed in Stephen King books and let it run, suddenly it outputs phrases, wording, place names, character names, adjectives all from Stephen King's repertoire. Is this a 'general language model'? Should this by copyright exempt? I don't think this is transformative enough at all. I've just mangled copyrighted works together, probably not enough to stand-up against a copyright claim.
I think people use AI and ML as buzzwords to try and obfuscate what's actually happening. If we were talking about AI and ML that doesn't need training on any licensed or copyrighted work (including 'public domain') then we can have a different conversation, but at the moment it's obscured copyright theft.
I can agree it's obscure in the sense that we shrug when asked about how it works. If you specifically train a model to mimic a specific style I can get behind it leaning more towards theft, or at least being immoral regardless of laws.
If you train a model to replicate 10000 specific artists, I could also get behind it being more like theft.
But if the intention was to train with random data (and some of it could be copyrighted) just like your polaroid example to generate anything you want, I'm not so sure anymore.
I feel the intent is the most important part here. But then again I don't know the intent behind these companies, and I guess you don't either. Maybe no single person working in these companies know the intent either.
It also gets murky when you have prompts that can refer to specific artists and when people who use the models explicitly try to copy an artists style. In the case of stable diffusion, if the CEO's to be believed the clip model had learned to associate images of greg ruktowski and other artists to images that were not theirs but in a similar style[0]
Even murkier is when you have a base model trained on public data, but people finetune at home to replicate some specific artist's style.
> If I scanned a thousand polaroid pictures, and took their average RGB values and created a LUT that I could apply to any photograph to make it look "polaroidy" - would that be learned?
Equating human cognition with machine algorithms is the root of the issue, and a significant part of its "legitimacy" comes from the need for "AI" companies to push their products as effective, and there's no better marketing than to equate humans to machines. Not even novel.
You can make out the two original copyrighted pictures in that case, and all you did was using 50% opacity which might not be very transformative, so probably?
In my mind (and I suspect others too) in machine learning context, statistical inference and learning became synonymous with all the recent development.
The way I see it, there's now a discussion around copyright because people have different fundemental views on what learning is and what it means to be a human that don't really surface.
> all this discussion on copyrights in the age of AI.
copyright is a thing, AI do not change that.
> does not 'steal' or and reproduce our code - it simply LEARNS from it as a human
And here we have the central problem, does it act like a human or does it not act like a human? Humans copy things they learn all the time, some of us know various songs by heart, others will even quote entire movies from memory. If AI can learn and reproduce things like humans do then you need to take steps to ensure that the output is properly stripped from any content that might infringe on existing copyrighted works.
There is a definite difference between singing a song while walking down the street and writing down the lyrics, putting it in a database, claiming it’s my content and then selling it on, even if it’s slightly rehashed.
I want that. I very much want someone to take one of the Windows code leaks, use it to train a LLM, and then make a fork of ReactOS with AI-completed implementations of everything ReactOS hasn't yet finished. Because then we could find out if Microsoft really believes that LLMs are fair use:)
"Learning" is a technical term, AI doesn't really learn the same way a human does. There is a huge difference between allowing your fellow human beings to learn from you and allowing corporations to appropriate your knowledge by passing it through a stochastic shuffler.
Copilot is run by a corporation, and the model is owned by the corporation - despite being trained on open source data.
In general individuals will have problems with the first L of LLMs - unless the community invents a way to democratise LLMs and deep learning in general. So far deep learning space a much less friendly place for individuals than software was when ideals of open source movement were formed.
A full LLM is too expensive for individuals to train, but LoRAs aren't.
There are multiple open source LLMs out there that can be extended.
We can already see it in AI art scene. People are training their own checkpoints and LoRAs of celebrities, art styles and other stuff that aren't included in base models.
Some artists demand to be excluded from base model training datasets, but there's nothing they can do against individuals who want to copy their style - other than not posting their art publicly at all.
I see the same thing here. If your source code is public - someone will find a way to train an AI on it.
No disrespect but I am baffled by your statement that it learns, even to go so far as to say as a human coder would learn.
I don't really want this to comment to be perceived as flame bait (AI seems to be a very sensitive topic in the same sense as crypto currency), so instead let me just pose a simple question. If Copilot really learns as a human, then why don't we just train it on a CS curriculum instead of millions of examples of code written by humans?
I think the comment was trying to draw the distinction between a database and a language model. The database of code on GitHub is many terabytes large, but the model trained on it is significantly smaller. This should tell us that a language model cannot reproduce copyrighted code byte for byte because the original data simply doesn't exist. Similarly, when you and I read a block of code, it leaves our memory pretty quickly and we wouldn't be able to reproduce it byte for byte if we wanted. We say the model learns like a human because it is able to extract generalised patterns from viewing many examples. That doesn't mean it learns exactly like a human but it's also definitely not a database.
The problem is that in reality, even though the original data is gone, a language model like Copilot _can_ reproduce some code byte for byte somehow drawing the information from the weights in its network and the result is a reproduction of copyrighted work.
I see what you're going for, and I respect your point of view, but also respectfully I think the logic is a little circular.
To say "it's not a database, it's a language model, and that means it extracts generalized patterns from viewing examples, just like humans" to me that just means that occasionally humans behave like language models. That doesn't mean though that therefore it thinks like a human, but rather sometimes humans think like a language model (a fundamental algorithm), which is circular. It hardly makes sense to justify that a language model learns like a human, just because people also occasionally copy patterns and search/replace values and variable names.
To really make the comparison honest, we have to be more clear about the hypothetical humans in question. For a human who has truly learned from looking at many examples, we could have a conversation with them and they would demonstrate a deeper sense of understanding behind the meaning of what they copied. This is something a LLM could not do. On the other hand, if a person really had no idea, like someone who copied answers from someone else in a test, we'd just say well you don't really understand this and you're just x degrees away from having copied their answers verbatim. I believe LLMs are emulating this behavior and not the former.
I mean, how many times in your life have you talked to a human being who clearly had no idea what they were doing because they copied something and didn't understand it all? If that's the analogy that's being made then I'd say it's a bad one, because it is actually choosing the one time where humans don't understand what they've done as a false equivalence to language models thinking like a human.
Basically, sometimes humans meaninglessly parrot things too.
> why don't we just train it on a CS curriculum instead of millions of examples of code written by humans?
I've never studied computer science formally but I doubt students learn only from the CS curriculum? I don't even know how much knowledge CS curriculum entails but I don't for example see anything wrong including example code written by humans.
Surely students will collectively also learn from millions of code examples online alongside the study. I'm sure teachers also do the same.
A language model can also only learn from text, so what about all the implicit knowledge and verbal communication?
What they are saying is that if you’ve studied computer science , you should be able to write a computer program without storing millions or billions of lines of code from GitHub in your brain.
A CS graduate could workout how to write software without doing that.
So they’re just pointing out the difference in “learning”.
LLM's are not storing millions or billions lines of code, and neither do we. Both store something more general and abstract.
But I'm saying there's a big difference between a CS graduate and some current LLM that learns from "the CS curriculum". A CS graduate can ask questions, use google to learn about things outside of school, work on hobby projects, study existing code outside of what's shown in university, get compiler feedback when things go wrong, etc.
All a language model can do is read text and try to predict what comes next.
> it simply LEARNS from it as a human coder would learn from it
Does it though? It "learns" correlations between tokens/sequences. A human coder would look at a piece of code and learn an algorithm. The AI "learns" token structure. A human reproducing original code verbatim would be incidental. AI (language model, at least) producing algorithm-implementing code would be incidental.
But.. to be clear what you can and can't do with certain code depends on the license.
Imagine code that is "open source" as in openly visible and available, yet the license explicitly forbids the use of it to train any AI/LLM.
Now how could the creator enforce that? Don't get me wrong, I am aware that the enforcement of such licenses is already hard (even for organizations like the FSF).. but now you are going up against something automated where you might not even know what exactly happens.
I'm in several communities for smaller/niche languages and asking questions about things that have few sources make it much more clear that it's not "learning" but grabbing passages/chunks of source. Maybe with subjects that have more coverage it can assimilate more "original" sounding output.
I would have no problem if such AI systems are also completely open source, can be run by me on my system and come with all models to use them also easily available (again in some form of opensource license). I genuinely don't see that happening in the future with BigTech. As such, as a proponent of FSF GPL philosophy, I have no interest in supporting such systems with my hard work, my source code. So yes, I do consider it stealing - my hard labour in any GPL opensource work is meant for the public good (for example, to preserve our right to repair by ensuring the source code is always available through the GPL license). Any corporate that uses my work, for profit, without either paying me or blocking the public good that I am trying for is simply exploiting me and the goodwill of others like me.
> it simply LEARNS from it as a human coder would learn from it
What humans do to learn is intuitive, but it is not simple. What the machine does is also not simple, it involves some tricky math.
Precisely if the process was simple, then it could be more easily argued that the machine is "just copying" - that is simple.
There's a lot of nuance here.
What the machine is doing "looks similar to what humans do from the exterior", the same way that a plane flying "looks similar" to a flying bird. But the airplane does not flap its wings.
> kind of irrational and antithetical to open source ideas
Plenty of people already argued that LLMs don't actually learn like a human. However, you should keep in mind the reason why clean-room reverse engineering exists: humans learn from source material. FLOSS RE projects (e.g. nouveau) typically don't like leaks, because some contributors might be exposed to copyrighted material. Sometimes, the opposite happens: people working on proprietary software are not allowed to see the source of a FLOSS alternative.
> it simply LEARNS from it as a human coder would learn from it.
It doesn't LEARN anything, let alone like a human coder would. It has absolutely zero understanding. It's not actually intelligent. It's a highly tuned mathematical model that predicts what the next word should be.
If you were to learn a phrase that insulted the king in Thai, and said it in Thailand, you would end up in jail. Doesn't matter if you understood what the phrase said. Ignorance doesn't make you immune to consequences.
Copilot does not steal. Copilot does not learn. If you want to apply these concepts to LLMs, first prove how an LLM is human and then explain why it doesn’t have human rights.
Rather, Copilot is a tool. Microsoft/ClosedAI operate this tool. Commercially. They crawl original works and through running ML on it automatically generate and sell derivative works from those original works, without any consent or compensation. They are the ones who violate copyright, not Copilot.
Your comment implies that we’re in some age of AGI, but we’re not there yet. Some argue that we’re not even close, but who knows, that’s all speculation.
> it simply LEARNS from it as a human coder would learn from it.
The LLM doesn’t learn, the authors of the LLM are encoding copyright protected content into a model using gradient decent and other techniques.
Now as far as I understand the law, that’s OK. The problems arise when distribution of the model comes into play.
I’m curious, are you a programmer yourself? Don’t take this the wrong way, but I want to understand the background of people who coming to the kind of conclusion you seemed to arrive at about how LLMs work.
Whether an LLM actually learns is completely tangential to the topic at hand. A human coder who learned from copyrighted code and then reproduced that code (intentionally or not) would be in violation of the copyright. This is why projects like Wine are so careful about doing clean room implementations.
As an aside, it seems really strange to invoke "open source ideas" as an argument in favor of a for-profit company building a closed source product that relies on millions of lines of open source code.
It’s also fair to say that a lot of this carefulness has probably made life difficult for the developers of wine, but they wanted to avoid Microsoft’s legal team. So they respected the copyright laws.
> IMHO desire to prevent learning from your open source code seems kind of irrational and antithetical to open source ideas
You may be right that this is antithetical to "open source" ideas, as Tim O'Reilly would've defined it - a la MIT/BSD/&c., but it's very much in line with copyleft ideas as RMS would've defined it - a la GPL/EUPL/&c. - which is what's being explicitly discussed in this article.
The two are not the same: "open source" is about widespread "open" use of source code, copyleft is much more discerning and aims to carefully temper reuse in such a way that prioritises end user liberty.
I still like Gitlab a lot and use it wherever I can. I think that it was once talked about very favorably on HN, but it seems to have lost its popularity after its IPO (or even earlier).
Codeberg seems like a decent host for Free Software projects. There is also Sourceforge, but I don't know if people have already forgiven and forgotten the mistakes made in the past.
Closed source. I can't see the point of publishing anything innovative if it is just going to be vacuumed up by some giant corporation and sold back out to users at $10 a month.
the problem with this approach is that humans suck and this is an invitation for trolls to upload your code to github. i don't even know how you're supposed to solve this.
you could proactively scan github for your code and try to get them to purge it if you find it i suppose, if that would remove your code from copilot. but even that is not a great solution because you would need to prove you're the actual author, and github would probably need to be involved in building a mechanism to do so, but they don't give a shit.
i think the reality is the LLMs have eroded copyright protections and trying to fight it isn't likely to pay off
I think fighting this is very worthwhile. For example, I think the fight is the reason why The Stack, dataset produced from GitHub, has a form to request to remove code.
do you know if the stack anticipated people who will claim ownership of and request removal of repositories that they do not own? it seems like complying would require github to expend a lot of effort to combat abuse.
It's very simple. If it's open source and the license allows re-publishing code (with copyright information intact), people can republish the code. You don't get to discriminate, cherry-pick, etc. Depending on the license, there may be lots of rights and requirements. Especially with GPL style licenses. Especially AGPLv3 seems to impose a lot of restrictions. Which is a reason for that license to not be popular with corporations. But even that license allows republishing code as long as you preserve license and copyright information.
This generally is the whole point of open sourcing code: to allow others to republish with or without modifications. If you don't want that, don't open source your code. Github is a perfectly valid and legal place to publish code. Lots of people have done so.
The jury is still out on whether using open source code as training data is fair use or constitutes a copyright violation. But the safest assumption at this point is that it is a combination of fair use and hard to prove that any violation happened to begin with (i.e. good luck stamping your feet in anger in a court room).
Which will no doubt make for some fun court cases but will also take many many years to come to any kind of conclusion. I would recommend not to get your hopes up on that. Historically, copyright law has not been updated a lot to deal with any kind of technical change. And that's just inside the US. The world is bigger than that of course. A safe assumption is that judges will seek to interpret any new cases under centuries of existing interpretations. Because that's what they do. Any changes to the law would have to go through politicians. And then the interpretation of that relative to other laws is again up to judges. Which is why we have such wonderful things as DMCA and a few other attempts to close loopholes in copright law.
Of course time is not your friend here. By the time this rolls through the courts some years down the line, existing practice will have evolved to be hopelessly dependent on language models for just about anything. Including interpreting the law (e.g. chat gpt seems to be acing exams) and doing all sorts of complicated things in science, engineering and technology. So, any legal outcome that that is somehow illegal is going to be combination of unpopular, economically damaging, and therefore unlikely. Think big corporations ki
That's a big if. Also, this is nearly forty years ago. With chip designs, there was a clear cut case of people taking each other's designs and some of the parties were billion $ chip companies being very grumpy about other companies taking their heavily patented solutions and copying those. Which translates into a lot of political pressure and lobby power and some political action. I don't see that happening for language models.
Most of the people complaining seem to be a narrow subset of open source developers that like open source but not people doing anything for profit with their source code. I.e. not your typical multi billion dollar companies that are worried about their IP getting stolen. So, not a lot of lobbying power or ability to get organized.
>It's very simple. If it's open source and the license allows re-publishing code (with copyright information intact)...
If it is that simple then clearly Copilot need to include the licence of the code when they republish it. Changing GPL'ed code doesn't make it not-GPL code at any point. If it started out as GPL it is still GPL later in the line when a user sees it in Copilot.
See, if a comment like that crops up on a civilised society like this, it demonstrates that the barren wastelands of “the outside world” will have similar examples.
Please upload all open source code to everywhere that will accept it. Make everything indestructible. Make it as easy as possible to find an authentic copy. Stick every bit of code with license that allows it in every free and proprietary neural network.
Did you write the absolute best implementation of X? I want to see it everywhere. Everywhere. In every single place where X is needed or discussed. Where all fine X are sold. Don't you? Or do you genuinely want to narrow the people who see your impl by some signifnicant percentage because you got frustrated with the ugly capitalism of a recent distribution mechanism?
If I invented Golden rice[0] I'd like to think I'd allow it to be sold at Panda Express and all the other evil capitalist chains, not just the local co-op whose business practices I prefer.
This may seem an extreme comparison, but to one who deeply cares about Free Software as defined by the FSF, proprietary software is bad. That's the difference between people who prefer Apache/BSD/MIT licenses versus those who prefer the (A)GPL.
It will not change anything. AI farms will git-clone from any public repository. Someone concerned should just patent-troll any software that may or may not use a copilot-ed part of their code.
Of course this isn't just a problem with copyleft licenses on github, but also with non-open code on github. Only there the problem may be less visible.
Ideally, github should check the license of the code it's using to feed copilot, and only use code with permissive licenses.
If it was possible to point to where the code is actually stored in Copilot (i.e. run strings on the server and it spits out the copylefted code), I would be a lot more sympathetic to the view that LLMs are stealing code. Even if you decode the weights from floats to strings, you won't find the strings of code stored anywhere. It's probably not correct to say it's "learning" from code the same way humans are, but it is "learning" from code in the way LLMs learn.
Fundamentally it seems to me that Copilot truly does synthesise code from some store of knowledge (even if it's hard to understand what this store of knowledge is), and the problem is that it's synthesising code identical to existing code. There are legal tools and also rhetoric that are designed for dealing with this problem of "synthesising something that is identical to an existing thing", and it's different tools and rhetoric from the ones we have for dealing with the problem of "stealing or copying existing things". It's valid to have an issue with with Copilot ingesting your code, but unfortunately people are largely using tools and rhetoric from the latter category to approach the issue, and that slight misapplication is causing their issues to fall on deaf ears a lot of the time.
What happens if I create a 10 lines function character by character identical to some proprietary or GPLled piece of code, without ever looking at that code nor knowing that it existed?
I expect that the copyright holders could reach to me, tell me that my code is identical to theirs, believe or not that I independently created the same code, and at least ask me to stop distributing it.
Of course
t += total(e)
is easier to defend than
monthlyTotal += dailyTotal(expenses)
because the chances that I didn't copy their code get slimmer and slimmer as the names of the identifiers and the structure of the code get more complex.
If I actually looked at their code, it would be wiser from me to at least change the names, maybe also a little bit the structure of the code. That's the difference between being inspired by something and copying it.
I know the law doesn't work this way, but wonder if code that can be "learned" this easily should really be copyrightable, etc. These LLMs don't keep a copy of some code to paste in verbatim.
LLMs totally do, and the fact can be confirmed by experiment. See "Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4": https://arxiv.org/abs/2305.00118
> What happens if I create a 10 lines function character by character identical to some proprietary or GPLled piece of code, without ever looking at that code nor knowing that it existed?
Then you're in the clear, though you may need to convince a court that that's what happened. (Patents and trademarks could still be issues, but there's no copyright issue)
Now, I'm not a lawyer, so this isn't legal advice, but...
A derivative work is not fair use. If you end up with a significant portion of another person's program in yours, (such that a substantial portion of your program is in some way related to their program), that will likely be a derivative work - but the definition of a derivative work depends on your jurisdiction and use-case. If you're unlawfully using the source material to produce a derivative work, you cannot copyright that derivative work under 17 U.S.C. § 103(a), and under the same section, you can only copyright your modifications, not the original.
It would be hard to argue fair use in this case; fair use only really applies for parodies/criticism, reporting, and scholarly works - and generally that's an affirmative defense, rather than an express or implied right you have.
Honestly, Copilot is difficult because Copilot can't be the author of the code; the person who used Copilot is the "author" of the code, and I think they'd be the ones liable for copyright infringement if copyrighted content ends up in their code.
To argue someone performed copyright infringement, all you need is to prove (1) a valid copyright exists; (2) that the person had access to the work; (3) the person had the opportunity to steal the work; and (4) that protected elements of the work had been copied (afaik generally under a "substantially similar" standard). Copilot offers an easy way to check both (2) and (3) - a copyright holder could argue that people had access to their code through Copilot, and that Copilot offered an opportunity to steal the work.
> If it was possible to point to where the code is actually stored in Copilot (i.e. run strings on the server and it spits out the copylefted code), I would be a lot more sympathetic to the view that LLMs are stealing code.
I think of it the other way around: I'd be more trusting that LLMs aren't a problem in that sense, if the big commercial entities linked to them used their own code to train them as well as public repositories.
MS's codebase must contain much good training material, unless they think their own code is crap and are embarrassed to let the AI look at it!
I am still undecided, but erring in the direction of not wanting my stuff to be used to train them, though I've long since passed Douglas Adam's "35 year's old" tech barrier so maybe I'm just not liking change…
For now at least I won't be putting my own code in services like GitHub, then at least if someone else does or an LLM is trained on public sites generally, I've not explicitly agreed to anything that says the company can use my code that way - which you do (they say, I believe there is at least one court case brewing on the matter) when you sign up to the services (agreeing to their terms in the process) and use them that way.
I feel like I’m reading the exact same sentence that someone posted about if StableDiffusion included images or was just “learning patterns” and totally generative from 6 to 8 months ago.
People argued, oh so hard about it.
“But stable diffusion is GB in size, how can it have embedded full images of a dataset that is so much larger? It’s not possible.”
…but, it turns out it is possible (1) and it does, in fact, have embedded full copies of the source images.
Now. Let’s talk about code…
> and the problem is that it's synthesising code identical to existing code
There’s a word for that. It’s called copying.
Copying fragments, copying full text. It’s a black box, it spits out content that is indistinguishable (again, see stable diffusion, a lossy jpg of an image is the same image even when it is not bit-for-bit identical) from the input.
That’s copying.
The black box might be a neutral network, or a python script that reads a file from disk. The process is irrelevant.
If you memorise a block of code and type it out by hand with no reference, you are copying it.
These models copy code.
These models have embedded full text copies of some code.
What we do with that is an ethical question, but that it is true is not in dispute. It is true. It’s documented.
If you think these models are not copying at least some training data as output you are factually, and provably incorrect.
Does this storage argument also go for humans? If I ask a painter to paint paintings they've seen before, what will be their rate of copying the painting exactly?
If you transcribe a copyrighted book by hand, that doesn't give you the right to publish it. I don't think being a human currently gives you a legal loophole to copy works so why make the comparison?
I don’t care if there’s a human in the box. If the box spits out training input as output it is copying it.
It doesn’t matter if there is a human doing it or not.
For your supposition to work, the input to the box would be only an abstract summary of the logical steps, and the output an exact copy of some other thing that was never an input.
In that case, yes, it would not be copying.
..but, is that the case? Hm? Is that what we care about? Is it possible to randomly generate the exact sequence of text with no matching training input? Along with the comments?
It seems fabulously unlikely, the point of being totally absurd.
I'm a proponent of not restricting (well, or trying to restrict) machine learning models and not considering them a lossy database but it must be said here if humans can recreate copyrighted works from memory and publish them, they are in trouble too.
I think the best argument here is that having the work in memory is not illegal, and human brains are not bound to copyright even when they can also be considered lossy databases. The question is where do we draw the line for a lossy database.
I agree. The problem is that a human has ethical deterrents to avoid copying data while a machine doesn’t, so we have to rely purely on legal incentives to avoid copies from being produced.
Wrong. Literally the whole thing about law - and especially about intellectual property laws - is that process is as much, if not more relevant, than the outcome. This is why "code as law" efforts are plain suicidal. This is why you can't just print out a hex dump of a pirated MP3 file and claim it's not copyright violation because it's just a long number that your RNG spit out - it would've been a good argument if your RNG actually did that, but it didn't, and that's what matters.
This is what it means when we say that, for just about everyone except computer scientists, bits have colour[0]. Lawyers and regulators - and even ordinary people - track provenance of things; how something came to be matters as much, and often more, than what the thing itself is.
This is what makes generative AI case interesting. They're right there in the middle between two extremes: machines xeroxing their inputs, and humans engaging in creative work. We call the former "copying", and the latter "innovation" or "invention" or "creativity". The two have vastly different legal implications. Generative AI is forcing us to formalize what actually makes them different, as until now, we didn't have a clear answer, because we didn't need one.
The pirate bay founders made the argument that process was necessary and lost fairly big. They argued that the process dictated that the prosecutor had to first prove that a copy had been made, and prosecute that, before they could argue that the pirate bay somehow helped with that crime.
The court did not agree. They looked instead towards an anti-biker gang law that illustrated that a biker bar can be found guilty of assisting with gang crime, even if no specific crime can be directly associated with the bar.
The defense team argument - that prosecutors need to prove that a crime had occurred - failed. The courts only require that the opposite is not believable, which given all the facts around the case was deemed sufficient. In that question the process doesn't matter. If the court do not think it believable that copying has not occurred, any argument about "machines xeroxing their inputs and humans engaging in creative work" will be ignored.
Funnily, the copyright argument is shifting towards “if resemblance is substantial, that’s infringing anyway”, circumventing whole discussions around “is it infringing if it’s called learning” arguments.
the human painter will undoubtedly be able to put out 1 billion exact copies of paintings based on their memory and skills in under 10 seconds so there is no reason to consider any difference in the cases.
Why is registering a copyright relevant in cases of copyright infringement? Copyright infringement is infringement of the right of someone else against copying.
Are you asking if it is okay for humans to plagiarize or otherwise violate copyright? For both the human and the algorithm, the answer is: it depends. Yes?
Are you trying to make money off of it? Is it for personal use? Was it protected or public domain?
It just all depends on the circumstances. Human or not.
“It depends on the circumstances” seems to me like cover for “we made up some random rules that made sense at the time”.
If you cover a song on YouTube, you can apparently be demonetised or attract copyright strikes from the original artist. But in software, you tell me an algorithm and I code it up, that’s an original work.
Oh, I used to be a lawyer. "It depends" is just always the answer, whether we like it or not. And, yes, it is pretty much because of “we made up some random rules that made sense at the time”.
Close to zero? It's too hard to create an identical picture from memory, even with the original the fakes are not bit-perfect. Computers, on the other hand, are great at copy&paste
Except they're not far? We're not talking about average images with 3 eyes or similarly obviously wrong code, the original comment is specifically about storing perfect copies. Human brains can't store perfect copies (and then reproduce), especially at scale
I'll see if I can dig some up, but there have been similar examples to the image reproduction case that have come out from codepilot, all it took was using some more unique method names (i.e. ones that were very specific to a particular code base) and oh look there comes the "totally not copied, honest gov'nor" code.
Any company that is using copilot or similar is walking right in to a massive legal minefield. Any engineer that chooses to use Copilot without explicit approval from the company should seriously worry about what target they're putting on their back. You don't want to be putting yourself in a position where your employer can say "They did this without our approval" or worse "Despite explicitly having told them not to", should there end up being legal action. Someone is going to establish precedence one way or another at some point, don't let it be you.
"I'll see if I can dig some up, but there have been similar examples to the image reproduction case that have come out from codepilot, all it took was using some more unique method names "
These assertions have very little weight - what matters is what the law says. And for now, in the jurisdictions that matter, the law is still silent. Maybe it will catch up via either legislation or case law, or maybe it won't. But for now there is no legal consensus on what these new things are doing.
Of course there is a moral/ethical component to this, and individuals are obviously free to hold personal views. But unless some global working consensus forms (unlikely) then I don't see how change will come from that.
I suspect that it's going to be difficult for politicians to find the will to do something about this issue, and then to get their heads around it enough to make good laws.
The subject of the thread is "Will my code be regurgitated if I put it on Github ? Should I avoid putting my code on Github if I don't want it copied?"
The legality aspect that you are injecting into the discussion is irrelevant
If y = f(x, T) when y is a member of T, prove f() doesn’t copy a value from T.
This is possible. For example, many functions f(0) return 0; if we can derive a function g(x) that generates y without access to T (ie. the training data) then we can plausibly assume that f(x, T) might indeed not be copying from T.
…but can we do so?
If so, we must admit that it is possible that f(x, T) = g(x) and no copying is taking place.
So, in the past 50 years of prior work can we come up with an example of a function g(x) that generates the exact code we see coming out of these LLMs?
I’ll grant, it’s not impossible.
For example, if stable diffusion generated sin wave patterns or fractals that exhibited “deep complex structure” and it happens that some artist had written code to do that exact thing, there would (I believe) be a fair case that stable diffusion was not copying the artist, it was simply a parallel implementation of the same generative code.
However, now my scepticism kicks in.
For code written by hand, that was not generated, we are suggesting that a LLM is a parallel equivalent implementation of a human mind, and it can, with purely “learnt patterns” replicate not only the intent and structure of known code, but the exact code itself, repeatedly.
I’ll grant. It’s not impossible, and it’s very difficult to prove, but it seems fabulously, unbelievably, extraordinarily unlikely that a clean room reimplementation would be identical, repeatedly.
We’re entering into the domain of assigning probability and then doing a zero knowledge proof here, but the solution is easy.
Just prove by counter example.
Train a model that generates such code without the code as training data.
You have now won the argument.
…on the bright side, you’ve also solved the 10 million dollar question of “how do I train my LLM when I don’t have enough training data” (because that’s what you just did; train it to do something without training data), so you’re now rich and give zero ducks what I think. Congrats.
> For example, if stable diffusion generated sin wave patterns or fractals that exhibited “deep complex structure” and it happens that some artist had written code to do that exact thing, there would (I believe) be a fair case that stable diffusion was not copying the artist, it was simply a parallel implementation of the same generative code.
I understand your pseudo equations and I understand [some] engineering.
Why are artists writing code? Is that to show the parallel of if artists did write code?
Unless artists do really write code but that’s beyond the scope of what I’m asking.
The point was that some artist may have created a work representing a complex fractal, perhaps through code since it can be quite difficult to manually compute some fractal patterns.
And that in that case, g(x) that can produce a perfect copy of the artist's work without having the image in the training set provably exists, and is the actual mathematical fractal function itself.
The law is generally less concerned with how something happened and more concerned with what happened. So I would suspect that the legal frameworks will be more restrictive towards code that is reproduced verbatim, regardless of if the code is directly copied or LLM generated.
Basically if you take something then feed new info into it is that new thing a new thing or a derivative of that work or other? With these models the data and mathmatical spline is so smeared across thousands of endpoints it is hard to tell. However, I do not think the courts have to think about the details of how the models work. They can do something else so a jury can get its head around it.
But basically the courts will probably simplify it into copyright things go in one side. Other things come out the other that sort of kind of resemble the original thing. Do not worry about the details of how it is done. Is that new thing owned by the original copyright holder? Or many holders, as it is smeared into a hundred other items that may or may not have copyright holders? Or is it a new work? Or is it a derivative work just colored?
This could go either way. As someone put it very nicely here a few weeks ago. These AIs are like the most amazing auto complete you have ever seen. Now in copyright if you make something and I independently come up with the same exact thing. Also I can prove that I am in very little trouble. Now in this case without that input code that AI model probably would not predict that exact string. But it in some cases exactly predict it again but can predict thousands of other things also. Is that prediction copyrightable? What if it predicts part of the code but with something else? I as a 3rd party who did not create the model and just used it and got the code what are my liabilities? Then if it is what are the consequences of that? The courts will have to decide eventually.
> “But stable diffusion is GB in size, how can it have embedded full images of a dataset that is so much larger? It’s not possible.”
> …but, it turns out it is possible (1) and it does, in fact, have embedded full copies of the source images.
The _not possible_ assertion is in response to people arguing that generative image models literally copy and thus "steal" the images in their training set or that such generative models "work" by infringing on said images. The assertion is not that the models cannot possibly contain _any_ copies of training data, the assertion is that the models are generalizing and can not contain a majority of the training data.
Now, let's look at some lines from your linked article titled "Paper: Stable Diffusion 'memorizes' some images, sparking privacy concerns", which happens to lead with scare quotes around "memorizes" and already hedges with "some":
> Researchers only extracted 94 direct matches and 109 perceptual near-matches out of 350,000 high-probability-of-memorization images they tested . . . resulting in a roughly 0.03 percent memorization rate in this particular scenario.
I do not consider a 0.03% rate of full copies particularly damning. Instead, it sounds like a flaw in training which could be addressed, which the article does in fact also mention being as a matter of overfitting to images that are over-represented in the training data.
> the 160 million-image training dataset is many orders of magnitude larger than the 2GB Stable Diffusion AI model. That means any memorization that exists in the model is small, rare, and very difficult to accidentally extract.
Which side's claims do you feel this line supports?
Generative text models should, in theory, also be generalizing and not memorizing. In practice, however, it does appear far easier to get Copilot and the like to spit out exact copies of code, complete with license text. Presumably, there's just not enough code out there to sufficiently generalize from, so we're seeing overfitting as a matter of course.
If you made 1,000,000 AI-generated MP3s available and 0.03% of them matched Sony’s copyrighted music catalog then you would be liable for $250,000 * 30 in statutory damages. Arguing that infringement doesn’t occur because the incidence rate is low is wallpapering over a serious problem.
Really depends on the details. If the extractable works align with songs that are over-represent in the training data, they may largely consist of performances of public domain compositions. And if their extraction also requires prompting which explicitly requests an infringing result, then the actual liability might be something you could mount a defense for.
Or hell, the extracted potentially infringing and over-represented material might all be pop music set to variations of Pachelbel's Canon, and I'd pay to see that lawsuit.
Details matter here. For a given musical performance, there are at least 3 copyrights in action:
1. The copyright on the composition. This can also include arrangements - for instance, Gershwin's original piano version of Rhapsody in Blue is now public domain, but the orchestral version everyone knows,
2. The copyright on the sheet music (the actual layout, spacing, editorial notes, things like that.. it's actually an insanely deep subject. I've got an 800 page book on the subject - which is referred to as music engraving, as up until about 40 years ago it was literally done by engraving the plates by hand. Much much harder problem than doing normal book-style text layout, as it's fully 2D, whereas text is basically 1d with occasional special cases. (NB: This copyright is really only relevant to the musicians, conductors, etc, but it does matter.
3. The copyright of the particular recording. This is the really relevant one. A 5 year old recording of a 500 year old work is very much under copyright.
If that was the case, they'd be financing this stuff. Afaik they're not. What if you cross-referenced the sony copyrighted catalogue with 1 million traditional/public domain songs? I'd guess a 0.03% would be a rounding error.
If Edison goes over to Tesla’s house and looks at the latest invention on Tesla’s workbench, then goes to his factory and builds the same thing and sells it, then Edison has stolen and copied Tesla’s invention.
If Edison breaks into Tesla’s house at night, takes* the invention off his workbench, goes to his factory and uses it as a reference to build and sell the same thing, then Edison has stolen and copied Tesla’s invention.
In both cases we can say “stolen and copied”, but actual sequence of events and the legal recourse that Tesla has is different in each case.
This difference matters because open source is like Tesla having his workbench in a public place where people are encouraged to watch him work, play around with the inventions, help him build, etc. In the “Tesla’s public workbench” situation, Tesla’s recourse is debatably impacted in the first scenario, but it is definitely unchanged in the second scenario. Arguing as though Copilot is doing “second scenario stealing and copying” rather than “first scenario stealing and copying” sidesteps that debate about whether open sourcing your code made it ‘up for grabs’ for natural and/or artificial intelligences to learn from and/or memorize. Now, it’s not clear what side will win that debate - I personally think “this has infringed intellectual property rights” has a decent chance of being the victor. But sidestepping that debate is setting off peoples’ “you’re trying to pull a fast one” alarm and making them unsympathetic. That’s what I’m getting at here.
*: to avoid “steal vs copy depriving of property”-type discussions, postulate that Edison returns the invention to the workbench before Tesla notices
As to the linked article, I would quote from the article itself: “[The paper] is dense with nuance that could potentially be molded to fit a particular narrative”. I could say more: both the article and the paper state there were zero byte-identical matches, and 203 direct or perceptual near matches out of 175 million generations (or out 160 million training images) is 0.0001%, which argues against this very strong claim of “factually and provably incorrect”. But I think the argument I will actually make is this: I said Copilot was synthesizing identical code from its knowledge base, which is completely compatible with the claim that it is outputting code from memory (in the human sense of memory). What it is not compatible with is the claim that it is outputting code from memory (in the computer sense of memory).
They do copy code sometimes but most of the code they generate is new, I think some (maybe not many but some) people have the impression that they literally search for similar code and copy it exactly 100% of the time which isn’t true
humans have learnt by looking at a few examples, and from there can code almost anything. Humans can also learn without examples, although it's harder. But I expect the 1st C programmers didn't have a K&R book to learn from.
AI instead has looked at all the existing code and it has learnt to do no more than what the existing code can do.
Humans make sense of the world with an extremely limited amount of information. I have not read all of GitHub. Comparatively, I have not read a fraction of it. And I could not read all of GitHub even if I wanted to.
However, I do not need to read all of GitHub anyway because, as a human, I am capable of understanding.
Current generation "AI" is not capable of understanding. Current generation "AI" merely computes the probability of any given word appearing next in this sentence based on an utterly absurd amount of raw data.
That is not what humans do at all. We do not need this information. We cannot even process it.
You make a logical leap. "Machines learn differently from humans, therefore they cannot understand". My definition for "understanding" isn't just "whatever humans (and nobody else) do".
We do not know whether or not that is what humans do or how qualitatively different it is to what humans do, because we don't know all that much about human reasoning works.
Put another way: Some Markov models are Turing complete, so "merely" computing probabilities can be Turing complete with only minor steps, so trying to downplay the potential capabilities of models like this by handwaving about things we don't know is foolish.
We don't need that scale of input, but we also don't know if LLMs need that much input to do well, or if our current training protocols are simply poor. With ongoing work on reducing the training cost, it is at a minimum clear that current training methods are far from optimal.
It sounds like what you are saying is that the AI cannot write code which would do something it hasn’t seen in the training set. That is how I interpreted the “it has learnt to do no more than what the existing code can do”. Do i understand you right?
If so, how do you know that? Are you talking about the limitations of a particular implementation or a limitation of all AIs as a concept?
> ‘There are legal tools and also rhetoric that are designed for dealing with this problem of "synthesising something that is identical to an existing thing", and it's different tools and rhetoric from the ones we have for dealing with the problem of "stealing or copying existing things".’
Copyright law applies in both cases. If you create work that’s substantially similar to an existing one, you’re risking copyright infringement even if none of the original work’s content was reproduced in the mechanical sense.
If this were not the case, why would cover bands pay the original artists for rights to the songs?
“I learned to play ‘Yesterday’ by heart” doesn’t mean you can do anything with the song without paying the Beatles. The same applies to machine learning, if all the model has learned is to imitate copyrighted works.
Yes, I do think that LLMs are probably doing copyright infringement. There is a long and rich heritage, in tech and elsewhere, of arguing that copyright infringement is not stealing, it is its own separate (though similar) crime and usually a lesser crime to some degree. Maybe my original point could be rephrased snarkily as “you may have a point but until you stop sounding like the RIAA calling everything ‘stealing’, a lot of people are going to ignore you”.
> If it was possible to point to where the code is actually stored in Copilot (i.e. run strings on the server and it spits out the copylefted code), I would be a lot more sympathetic to the view that LLMs are stealing code
If it was possible to point where the pixel values are actually stored in a jpeg file (i.e. run dd on the file and it spits out the pixel values), I would be a lot more sympathetic to the view that LLMs are not stealing code.
In the very next sentence after the one you quote, I say “Even if you decode the weights from floats to strings, you won't find the strings of code stored anywhere.”
If you decode the jpeg file you do get the pixel values.
The whole debate is that when you decode the weights (i.e. "run AI on some prompt") you do in fact get training code reproduced verbatim. The fact that we do not have tools to analyze this decoding function analytically is orthogonal.
Aren't we just talking about a novel compression technique here. Like there was a voice codec Google released a while ago that essentially modeled your voice and then fed a piano roll of what you said through it.
I don't really care whether there's a lossless, lossy, or AI compression of my stuff. I care that it's my stuff.
How is any of that relevant? If i memorize your work and dissect my brain and run strings on the output you won't find anything either. Same happens if you put it in a .zip file. But both, and the language model, can spit it back out and infringe on your rights.
While I sympathise with the remarks, I think these arguments are wrong.
In any programming language there are only that many ways to write something. For instance, iterate over a list of items and do some processing to them.
So when someone publishes their code under GPL, suddenly millions of projects that come up with exactly the same code can be in a violation?
I get this is complex, but copyrighting code is akin to copyrighting maths formulas.
Imagine if a company selling bottled water could also copyright water. That's very much what is happening.
Naomi Klein's recent essay about AI^[1] suggests that the real "hallucination" problem in AI is that it's promoters are not seeing the real world clearly. Some of her points about the effect of AI rollout on human employment may be germane to the present discussion.
You're preaching to the cult here. There is a marked lack of AI criticism and skepticism on this forum. There's a few are trying to gaslight skeptics (yes, gaslight) that current AI is close to being sentient, and/or humans are not much smarter than it, and/or we should consider and evaluate its output as if it were created by a human.
I don't know if it's laughable, or utterly terrifying giving such a powerful tool into the hands of reckless wannabe entrepreneurs.
It's hype for suckers intentionally created by the mega corps (remember "Sparks of AGI"?). The real goal is to get away with theft long enough until the regulators give up and accept their right to be above the law (aka "disruption"). And then there's the army of intentionally misguided youtube creators that want to get views by showing how they created a website without knowing html
Weasel wording to evade legislation. It's not an unlicensed taxi, it's ride sharing. It's not an illegal hotel, it's couch surfing. It's not code licence infringement because it's learning.
It's lawyers finding loopholes for finance to avoid expenses that gives them an edge over the poor suckers who play by the rules. The tech is just a tool to this purpose.
And most people forget TANSTAAFL. The costs for the cars and their infrastructure, the load tourists put on a place, and the effort for writing the code are still there. The "innovators" just found a way to make somebody else pay for what should be their cost center.
Isn't e.g. ChatGPT trained on data all over the internet? That means even if you upload your code to GitLab or your own public Gitea instance, it will still be used to train AIs?!
I don't see the point of this because quite frankly - if you want to prevent AIs using your data, you already lost that battle in the moment you uploaded your Code to the publicly accessible internet.
This will always be an issue as long as people can fork the code, so one might say we need a license that prevents a module from being used in ML training, better yet, we need a way for a commented line or something that'd pork the training pipeline if found in the source training, and removing it would violate the license.
495 comments
[ 3.8 ms ] story [ 339 ms ] thread> Is this a legal document?
> No, it isn’t. If the project is under an open source license, it means that everyone can share a copy – even on GitHub – of the licensed material under certain conditions. A license restricting this right wouldn’t be open source anymore. However, since GitHub may not respect the terms of licensed code that is hosted on their servers, not uploading the code of others there is, in fact, an ethical choice.
emphasis mine. It's a "please be nice", not a "I want to enforce things"
I respect developers right to put any restrictions on the code they share with the world. But I believe it should either be explicit or not restricted at all. Either write the license that says exactly what you want or otherwise don’t shame people into the desired behavior.
Edit: one could even add a more generic statement to the license stating that it’s forbidden to share the code on any platform that would use it to train their AIs per their ToS, so you don’t need to single out GitHub and potentially others in the future.
it is a wish. if someone says "please don't wear shoes in my home" i hope you would honor their very simple and understandable personal wish without setting up a contract for it?
i mean, just be a bit more human, please.
Stallman would prefer I not use any closed source software to read his blog, including OS, drivers, web browser, etc.
People routinely ignore unreasonable requests. Asking me to not wear shoes in your home is reasonable. Asking me not to give a copy to Joe after telling me I can give a copy to whomever I want is unreasonable.
We already established no one is forcing you and if you don't respect the author you don't get to be respected for your decision to ignore them and will earn snarky remarks. (and rightfully so, in my opinion)
This is a repetition of your claim, not an argument for it. Counterpoint: It's entirely reasonable to ask you not to give a copy to Joe.
(1) Though I don't know if this has been actually tested in court - courts in India have more freedom to broadly interpret social contracts like the GPL, unlike the US courts, and a positive outcome in favour of upholding the license even in such cases could be possible).
I disagree here. The idea, intent, philosophy is one (crucial) thing, the resulting practical artefact (here the license) is another. It works exactly as it was designed to work.
People/companies modifying GPL software for their own use (internal or external) without redistributing the software itself (so without requirement to redistribute the code) existed before SAAS grew on, only at the time, the small scale of this made it a bargain that was "interesting" only depending on one's capacity/hubris to maintain an internal fork on their own.
*aaS hugely tipped the scale, and the side effects, but the mechanics are the same.
And yes, that may not have been the original intent, and the AGPL is as valid a license as a reaction to provide a new tool more in line with the original intent, but that doesn't make the use of the existing GPL all within what it actually enables anyone to, invalid or unethical.
(but maybe only in a specific perspective of the framework of the original intent)
How is this case different?
It took me some time to get that this is a generic call (to be followed and reused), still with no clear ownership, rather than a specific claim to a specific code/project (or is it?).
>> No, it isn’t.
My limited understanding of international copyright law is that copyright disclaimers/licenses per Berne Convention are documents stating the author's wishes no different than this one. When a judge is going to be shown this page where the author states his explicit wish to not have his code uploaded code on Github, in direct conflict with his wishes stated in the LICENSE.txt file, I don't see how it will hold up in court as free software code.
Also don't forget to go back in time to let your former self know which hosting providers will violate your license.
I mean even doing that will not protect anybody's code, Microsoft doesn't care, they might also be scanning gitlab or bitbucket and doing model training on these. The only way to protect your code at that point is to stay closed source. It's as simple as that.
It's no different from all these models trained on content without permission, whether it be articles or photos,... until all these corporations are sued or copyright laws change to adapt ML, they'll train their model on content regardless of the its license since they are getting away with it.
It's pretty clear in the article, I hesitate just just cut and paste it here, but the idea is that Github is not respecting some terms of some licenses, so if you upload another authors code to github, you are exposing the author rights to github's depradations.
I agree with 'can', but I find 'should' a weird choice of words here.
I don't think we are better off by phrasing hyper-specific licenses (or laws).
Github has been consistently arguing that they're not bound by the distribution license when training their Copilot model, so I'm not sure what difference that would make.
Even without this, in terms of copyright, since Copilot doesn’t do what your public code does, and it only uses your code to train, it is a transformative use, and would be fair use. It’s possible that a court case will find otherwise, but I think that’s unlikely. The only case I think it will become disallowed, is if Congress passes a law about it.
If Congress does pass such a law, GitHub’s market power in this domain only goes up, since the EULA gives it the covenants.
Unless you can reproduce a substantial portion of a repo, I think it’s going to be an uphill battle to argue it isn’t fair use. Though I suspect Copilot’s suppression feature will make doing so impossible.
Either way, the whole copilot thing smells of 'the issue of copyright infringement is more copyright infringement' to me.
(And I don't see why we cannot add a clause saying that the source code is only meant for human developers and use of the source code in any machine learning system or to train any AI systems is prohibited without explicit case-by-case permission).
I think github, after stackoverflow, is the best thing that happened to developers.
> I think github, after stackoverflow, is the best thing that happened to developers.
I don't think github is bad tooling (there are worse), but the best thing for developer? I don't see anything on github that is greatly better than what we have in other services.
The only leap here is the fact that the programmer has outsourced the learning to a tool that does it for them, which they then use to do their job, just as before.
Nobody is arguing against uploading code. It's about Github/Microsoft specifically.
However, at a first glance, it still feels to me like an unavoidable reality that if you publish source code it'll eventually be ingested by Copilot or whatever comes next.
I mean, for the rest of the content all the new fancy LLMs have been trained with, there wasn't a Github equivalent. They just used massive scraped dumps of text from wherever they could find them, which most definitely included trillions of lines of very much copyrighted text.
In short: not only I don't really see an issue with Copilot-like AIs learning from publicly available code (as I described in the GP comment) but I also think if you publish code anywhere at all it's inevitable that it'll end up in Copilot, regardless of where you host it. If you want to make it more expensive for Microsoft to scrape it, sure, go ahead, but I don't think it matters in the long run.
I’d be quite careful with of this view.
By your logic, it should be ok to take the Linux kernel, copy it, build it, then sell it and give nothing back to the community that built it. Then just blame it on the authors for uploading it to the internet ?
So that’s not accurate.
Though, GitHub would do well to also bake-in approp attributions if a significant portion of the generated code is a copypasta.
In the case of ChatGPT-x, Open AI company which is disguised as a not for profit with a goal of producing ever more powerful models that may eventually be capable of replacing you at work while seemingly not having any plan to give back to those who’s work was used to make them insane amounts of money.
They haven’t even given back any of their research. So it’s ok to take everyone’s open source work and not give back is it ?
This isn’t some cute little robot who wakes up in the morning and decided it wants to be a coder. This is a multi-national company who has created the narrative you’re repeating. They know exactly what they’re doing.
The interesting debate should be what happens in the gray area, when you read a lot of code and learns patterns and ideas.
> This can lead to some copylefted code being included in proprietary or simply not copylefted projects. And this is a violation of both the license terms and the intellectual proprety of the authors of the original code.
If the author was a human, this would be a clear violation of the licence. The AI case is no different as far as I can tell.
Edit: I'm definitely no expert on copyright law for code but my personal rule is don't include someone's copyrighted code if it can by unambiguously identified as their original work. For very small lines of code, it would be hard to identify any single original author. When it comes to whole functions it gets easier to say "actually this came from this GPL licensed project". Since Copilot can produce whole functions verbatim, this is the basis on which I state that it "would be a clear violation" of the licence. If Copilot chooses to be less concerned about violating the law than I am then that's a problem. But maybe I'm overly cautious and the GPL is more lenient than this in reality.
We aren't talking verbatim generation of entire packages of code here, are we? Code snippets are surely covered under fair use?
...for "purposes such as commentary, criticism, news reporting, and scholarly reports"? Sure.
For a commercial product? Best check with your lawyer...
In general it is not fair use if you are using the material for the same scope as the original author[0] or if you are doing it just to namedrop/quote/omage the original.
It is possible to argue that a snippet can be too small to be protected, but that would not be because of fair use.
[0] Suppose that some Author B did as above and copied a snippet of code in their docstring to exlain buggy behaviour of a library they were reimplementing. If you are then trying to reimplement B's libary you can copy the same snippet B copied, but you likely cannot copy the paragraph written by B where they explain the how and the why of the bug.
I guess the likelihood decreases as the code length increases but the likelihood also increases the more constraints on parameters such as code style, code uniformity etc you pose.
If someone somehow managed to do that and then happened to have accidentally copied someone's code, how believable would their argument be?
No, and humans who have read copyrighted code are often prevented from working on clean room implementations of similar projects for this exact reason, so that those humans don't accidentally include something they learned from existing code.
Developers that worked on Windows internals are barred from working on WINE or ReactOS for this exact reason.
That's just copying with extra steps.
The way to do it legally is to have 1 person read the code, and then write up a document that describes functionally what the code does. Then, a second person implements software just from the notes.
That's the method Compaq used to re-implement the original PC BIOS from IBM.
wasn't it to have one person run tests of what happened when different things were done, and then write up a document describing the functionality?
In other words I think one person reading the code is still in violation?
https://en.wikipedia.org/wiki/Clean_room_design
>by reverse engineering and then recreating it without infringing any of the copyrights associated with the original design.
reverse engineering is not 'reading the code'.
Except that with AI we can more easily (in principle) provide provable provenance of training set and (again in principle) reproduce the model and prove whether it could create the copyrighted work also without having had access to the work in its training set
Maybe, but that would still be copyright infringement. See My Sweet Lord.
Not necessarily. If it's just a small snippet of code, even an entire function taken verbatim, it may not be sufficiently creative for copyright to apply to it.
Copyright is a lot less black and white than most here seem to believe.
Copilot similarly isn’t the one checking in the code. So it’s on each user. That said, Copilot at some point probably needs to add some type of copyright detection heuristics. It already has a suppression feature, but it probably also needs to have some type of checker once code is committed and at that point Copilot generated code needs to be cross-referenced against code Copilot was trained on.
But only snippets as far as I can tell.
This is the codeexample linked from the author:
https://web.archive.org/web/20221017081115/https://nitter.ne...
It is still not trivial code, but are there really lot's of different ways on how to transpose matrixes?
(Also the input was "sparse matrix transpose, cs_", so his naming convention especially included. So it is questionable if a user would get his code in this shape with a normal prompt)
And just slightly changing the code seems trivial, at what point will it be acceptable?
I just don't think spending much energy there is really beneficial for anyone.
I rather see the potential benefits of AI for open source. I haven't used Copilot, but ChatGPT4 is really helpful generating small chunks of code for me, enabling me to aim higher in my goals. So what's the big harm, if also some proprietary black box gets improved, when also all the open source devs can produce with greater efficency?
If a medical procedure is proven to be life-saving, what happens worldwide? Doctors are forced to update their procedures and knowledge base to include the new information, and can get sued for doing something less efficient or more dangerous, by comparison.
If you write the most efficient code, and then simply slap a license on it, does that mean, the most efficient code is now unusable by those who do not wish to submit to your licensing requirements?
I hear an awful lot of people complain all the time about climate change and how bad computers are for the environment, there are even sections on AI model cards devoted to proving how much greenhouse gases have been pushed into the environment, yet none of those virtue signalling idiots are anywhere to be seen when you ask them why they aren't attacking the bureaucracy of copyright and law in the world of computer science.
An arbitrary example that is tangentially related: One could argue that the company sitting on the largest database of self-driving data for public roads is also the one that must be held responsible if other companies require access to such data for safety reasons (aka, human lives would be endangered as a consequence of not having access to all relevant data). See how this same argument can easily be made for any license sitting on top of performance critical code?
So where are these people advocating for climate activism and whatever, when this issue of copyright comes up? Certainly if OpenAI was forced to open source their models, substantial computing resources would not have been wasted training competing open source products, thus killing the planet some more.
So, please forgive me if I find the entire field to be redundant and largely harmful for human life all over.
The problem here is that Microsoft is effectively saying, "copyright for me but not for thee." As long as Microsoft gets a state-enforced monopoly on their code, I should get one too.
If you don't "slap a license on it" it is unusable by default due to copyright.
This. People seem to forget that generative AIs don't just spit out copyrighted work at random, of their own accord. You have to prompt them. And if you prompt them in such a way as to strongly hint at a specific copyrighted work you have in mind, shouldn't some of the blame really go to you? After all, it's you who supplied the missing, highly specific input, that made the AI reproduce a work from the training set.
I maintain that, if we want to make comparisons between transformer models (particularly LLMs) and humans, then the AI isn't like an adult human - it's best thought of as having a mentality of a four year old kid. That is, highly trusting, very naive. It will do its best to fulfill what you ask for, because why wouldn't it? At the point of asking, you and your query are its whole world, and it wasn't trained to distrust the user.
If you, not I, uploaded my GPL'ed code to Github is the blame on you then?
Definitely not me - if your code is GPL'ed, then I'm legally free to upload it to Github, and to an extent even ethically - I am exercising one of my software freedoms.
(Note that even TFA recognizes this and admits it's making an ethical plea, not a legal one.)
Github using that code to train Copilot is potentially questionable. Github distributing Copilot (or access to it) is a contested issue. Copilot spitting out significant parts of GPL-ed code without attaching the license, or otherwise meeting the license conditions, is a potential problem. You incorporating that code into software you distribute is a clear-cut GPL violation.
If we think of Copilot as a (de)compression algorithm plus the compressed blob that the algorithm uses as its database, the algorithm is fine but the contents of the database pretty clearly violate GPL.
If I start creating a car by using a blueprint of Fords to create something at what point will it be acceptable? I'd say even if you rework everything completely Ford would still have a case to sue you. I can't see how this is any different. My code is my code and no matter how much you change it, it is still under the same licence as it started out with. If you want it not to be then don't start with a part of my code as a base. In my opinion the case is pretty clear: This is only going on because Microsoft has lots of money and lawyers. A small company doing this would be crushed.
In a lot of scenarios, there is an existing best practice or simply only one real 'good' way to achieve something - in those cases are we really going to say that despite the fact a human would reasonably come to the same output code, that the AI can't produce it because someone else wrote it already?
It's not a problem in practice. It only does so if you bait it really hard and push it into a corner, at which point you may just as well copy the code directly from the repo. It simply doesn't happen unless you know exactly what code you're trying to reproduce and that's not how you use Copilot.
It should be easy to check Copilot's output to make sure it's not copied verbatim from a GPL project. Colleges already have software that does this to detect plagiarism.
If it's a match, just ask GPT to refactor it. That's what humans do when they want to copy stuff they aren't allowed to copy, they paraphrase it or change the style while keeping the content.
https://github.blog/2022-11-01-preview-referencing-public-co...
If the data could only used by other open source projects, e.g. open source AI models, I don't think anyone would complain.
You could argue "well, but anyone can use the code on Github" and while that's technically true, it's obvious that with both Github and OpenAI being owned by Microsoft, OpenAI gets a huge competitive advantage due to internal partnerships.
People put code on github to be read by anyone (assuming a public repository), but the terms of use are governed by the license. Now you've got a system that ignores the license and scrapes your data for its own purpose. You can pretend it's human but the capabilities aren't the same. (Humans generally don't spend a month being trained on all github code and remember large chunks of it for regurgitation at superhuman speeds, nor can they be horizontally scaled after learning.)
You can still be of the opinion that this is fine, and I may or may not be fine with it as well, I just don't think the stated reason holds up to logic and other opinions ought to "baffle" you
> Copilot does not 'steal' or and reproduce our code - it simply LEARNS from it as a human coder would learn from it.
Not "the terms of use you agreed to allow them to do it". Different argument with different amount of merit in my opinion
For MIT licenses that's impossible currently because of the requirement to mention the authors.
Personally I have eschewed any personal use of Github since the MS aquisition and only ever use it where that's mandated by a client (so not my code). If you clone my code from elsewhere into a Github repo, that's just rude and contrary to me every intent and wish.
I think it's time to add a "No GitHub" clause as an optional add-on to the various open-source licenses.
Intellectual property is a nebulous concept to begin with, if you really try to understand it. There's a reason copyright claim systems like those at YouTube don't really concern themselves with ownership (that's what DMCA claims are for) but instead with the arbitrary terms of service that don't require you to have a degree in order to determine the boundaries of "fair use" (even if it mimics legal language to dictate these terms and their exemptions).
The problem isn't AI. The problem is property. Ever since Enclosure we've been trying to dull the edges of property rights to make sure people can actually still survive despite them. At some point you have to ask yourself if maybe the problem isn't how sharp the blade you're cutting yourself is but whether you should instead stop cutting. We can have "free culture" but then we can't have private property.
Also, AI doesn't learn "like a human". Neural networks are an extremely simplistic representation of a biological brain and the details of how learning and human memory works aren't even all that clear yet.
Open source code usually comes with expectations for the people who use it. That expectation can be as simple as requiring a reference back to the authors, adding a license file to clarify what the source was based on, or in more extreme cases putting licensing requirements on the final product.
Unless Microsoft complies with the various project licenses, I don't see why this is antithetical to the idea of open source at all.
If I took two copy-righted pictures and layered them on top of each other at 50% opacity. Would that be OK or copy right infringement?
AI models just use more weights/biases and more images (or any input).
if(nonfree_software){
// unhappy path
}
If I scanned a thousand polaroid pictures, and took their average RGB values and created a LUT that I could apply to any photograph to make it look "polaroidy" - would that be learned? Or the application of a statistical inference model? This alone is probably far enough abstracted to never be an ethical or legal issue. However, if I had a model that was only "trained" on Stephen King books, and used it to write a novel, would that be OK? Or do you think it would be in the realm of copyright infringement?
By your definition anything a computer does means it has learned it. If I copy and paste a picture, has the computer "learned" it while it reads out the data byte-by-byte? That sure sounds like it is "studying" the picture.
"AI" and "ML" are just statistics powered by computers that can do billions of calculations per second. It is not special, it is not "learning". To portray some value to it as something else is disingenuous at best, and fraud at worst.
In your stephen king example I would say it's still learned, because the "code" is a general language model that can learn anything. It's just you decided to only train it on stephen king novels. If you have an image model that trained 100% on public domain images and finetune it to replicate a specific artist's style I would personally think the finetuned model and its creator is maybe violating copyright.
But when it comes to learning I would say when you write a program whose purpose is to learn the next word or pixel, but it's up to the computer to figure out how to do that, the computer is learning when you feed it input data. It's the program's job to figure out the best way to predict, not the programmer. (it's not that black and white given that the programmer will also sometimes guide the program, but you get the idea)
When you write a program that does one or several things, it's not learning.
I think it's something to do with the difference between emergent behavior from simple rules and intentional behavior from complex rules.
If I created a program to read words from the input and assign weights based on previous words, I could feed in any data. Just like the polaroid example. (I suggested that the polaroid example was abstract enough not to be an ethical/legal problem because I believe it is mostly transformative, unless the colours themselves were copyrighted or a distinct enough work in themselves.)
Now If I only feed in Stephen King books and let it run, suddenly it outputs phrases, wording, place names, character names, adjectives all from Stephen King's repertoire. Is this a 'general language model'? Should this by copyright exempt? I don't think this is transformative enough at all. I've just mangled copyrighted works together, probably not enough to stand-up against a copyright claim.
I think people use AI and ML as buzzwords to try and obfuscate what's actually happening. If we were talking about AI and ML that doesn't need training on any licensed or copyrighted work (including 'public domain') then we can have a different conversation, but at the moment it's obscured copyright theft.
If you train a model to replicate 10000 specific artists, I could also get behind it being more like theft.
But if the intention was to train with random data (and some of it could be copyrighted) just like your polaroid example to generate anything you want, I'm not so sure anymore.
I feel the intent is the most important part here. But then again I don't know the intent behind these companies, and I guess you don't either. Maybe no single person working in these companies know the intent either.
It also gets murky when you have prompts that can refer to specific artists and when people who use the models explicitly try to copy an artists style. In the case of stable diffusion, if the CEO's to be believed the clip model had learned to associate images of greg ruktowski and other artists to images that were not theirs but in a similar style[0]
Even murkier is when you have a base model trained on public data, but people finetune at home to replicate some specific artist's style.
[0] https://twitter.com/EMostaque/status/1571634871084236801
You wouldn't. LUT would.
Since it’s data that should be cool right ?
In my mind (and I suspect others too) in machine learning context, statistical inference and learning became synonymous with all the recent development.
The way I see it, there's now a discussion around copyright because people have different fundemental views on what learning is and what it means to be a human that don't really surface.
copyright is a thing, AI do not change that.
> does not 'steal' or and reproduce our code - it simply LEARNS from it as a human
And here we have the central problem, does it act like a human or does it not act like a human? Humans copy things they learn all the time, some of us know various songs by heart, others will even quote entire movies from memory. If AI can learn and reproduce things like humans do then you need to take steps to ensure that the output is properly stripped from any content that might infringe on existing copyrighted works.
In general individuals will have problems with the first L of LLMs - unless the community invents a way to democratise LLMs and deep learning in general. So far deep learning space a much less friendly place for individuals than software was when ideals of open source movement were formed.
There are multiple open source LLMs out there that can be extended.
We can already see it in AI art scene. People are training their own checkpoints and LoRAs of celebrities, art styles and other stuff that aren't included in base models.
Some artists demand to be excluded from base model training datasets, but there's nothing they can do against individuals who want to copy their style - other than not posting their art publicly at all.
I see the same thing here. If your source code is public - someone will find a way to train an AI on it.
I thought that was a sarcastic remark, given the capitalization of 'learn', but followed by IMHO dispelled that part.
We have no idea how humans learn, and the 'AI' has a statistical approach, not much more than that.
Have you personally ever put out something in public domain?
I don't really want this to comment to be perceived as flame bait (AI seems to be a very sensitive topic in the same sense as crypto currency), so instead let me just pose a simple question. If Copilot really learns as a human, then why don't we just train it on a CS curriculum instead of millions of examples of code written by humans?
The problem is that in reality, even though the original data is gone, a language model like Copilot _can_ reproduce some code byte for byte somehow drawing the information from the weights in its network and the result is a reproduction of copyrighted work.
To say "it's not a database, it's a language model, and that means it extracts generalized patterns from viewing examples, just like humans" to me that just means that occasionally humans behave like language models. That doesn't mean though that therefore it thinks like a human, but rather sometimes humans think like a language model (a fundamental algorithm), which is circular. It hardly makes sense to justify that a language model learns like a human, just because people also occasionally copy patterns and search/replace values and variable names.
To really make the comparison honest, we have to be more clear about the hypothetical humans in question. For a human who has truly learned from looking at many examples, we could have a conversation with them and they would demonstrate a deeper sense of understanding behind the meaning of what they copied. This is something a LLM could not do. On the other hand, if a person really had no idea, like someone who copied answers from someone else in a test, we'd just say well you don't really understand this and you're just x degrees away from having copied their answers verbatim. I believe LLMs are emulating this behavior and not the former.
I mean, how many times in your life have you talked to a human being who clearly had no idea what they were doing because they copied something and didn't understand it all? If that's the analogy that's being made then I'd say it's a bad one, because it is actually choosing the one time where humans don't understand what they've done as a false equivalence to language models thinking like a human.
Basically, sometimes humans meaninglessly parrot things too.
This just means it's a really efficient lossy compression algorithm, not that it learns like a human.
I've never studied computer science formally but I doubt students learn only from the CS curriculum? I don't even know how much knowledge CS curriculum entails but I don't for example see anything wrong including example code written by humans.
Surely students will collectively also learn from millions of code examples online alongside the study. I'm sure teachers also do the same.
A language model can also only learn from text, so what about all the implicit knowledge and verbal communication?
A CS graduate could workout how to write software without doing that.
So they’re just pointing out the difference in “learning”.
But I'm saying there's a big difference between a CS graduate and some current LLM that learns from "the CS curriculum". A CS graduate can ask questions, use google to learn about things outside of school, work on hobby projects, study existing code outside of what's shown in university, get compiler feedback when things go wrong, etc.
All a language model can do is read text and try to predict what comes next.
Does it though? It "learns" correlations between tokens/sequences. A human coder would look at a piece of code and learn an algorithm. The AI "learns" token structure. A human reproducing original code verbatim would be incidental. AI (language model, at least) producing algorithm-implementing code would be incidental.
This is really not how LLMs work.
What humans do to learn is intuitive, but it is not simple. What the machine does is also not simple, it involves some tricky math.
Precisely if the process was simple, then it could be more easily argued that the machine is "just copying" - that is simple.
There's a lot of nuance here.
What the machine is doing "looks similar to what humans do from the exterior", the same way that a plane flying "looks similar" to a flying bird. But the airplane does not flap its wings.
> kind of irrational and antithetical to open source ideas
Open source ideas are not the only ideas in town.
It doesn't LEARN anything, let alone like a human coder would. It has absolutely zero understanding. It's not actually intelligent. It's a highly tuned mathematical model that predicts what the next word should be.
Rather, Copilot is a tool. Microsoft/ClosedAI operate this tool. Commercially. They crawl original works and through running ML on it automatically generate and sell derivative works from those original works, without any consent or compensation. They are the ones who violate copyright, not Copilot.
> it simply LEARNS from it as a human coder would learn from it.
The LLM doesn’t learn, the authors of the LLM are encoding copyright protected content into a model using gradient decent and other techniques.
Now as far as I understand the law, that’s OK. The problems arise when distribution of the model comes into play.
I’m curious, are you a programmer yourself? Don’t take this the wrong way, but I want to understand the background of people who coming to the kind of conclusion you seemed to arrive at about how LLMs work.
As an aside, it seems really strange to invoke "open source ideas" as an argument in favor of a for-profit company building a closed source product that relies on millions of lines of open source code.
Here is Microsoft doing as Microsoft does…
You may be right that this is antithetical to "open source" ideas, as Tim O'Reilly would've defined it - a la MIT/BSD/&c., but it's very much in line with copyleft ideas as RMS would've defined it - a la GPL/EUPL/&c. - which is what's being explicitly discussed in this article.
The two are not the same: "open source" is about widespread "open" use of source code, copyleft is much more discerning and aims to carefully temper reuse in such a way that prioritises end user liberty.
[0] https://www.sqlite.org/whynotgit.html
https://www.gitblit.com/
you could proactively scan github for your code and try to get them to purge it if you find it i suppose, if that would remove your code from copilot. but even that is not a great solution because you would need to prove you're the actual author, and github would probably need to be involved in building a mechanism to do so, but they don't give a shit.
i think the reality is the LLMs have eroded copyright protections and trying to fight it isn't likely to pay off
https://www.bigcode-project.org/docs/about/the-stack/
I am hopeful that we can make the removal request form the standard industry practice.
This generally is the whole point of open sourcing code: to allow others to republish with or without modifications. If you don't want that, don't open source your code. Github is a perfectly valid and legal place to publish code. Lots of people have done so.
The jury is still out on whether using open source code as training data is fair use or constitutes a copyright violation. But the safest assumption at this point is that it is a combination of fair use and hard to prove that any violation happened to begin with (i.e. good luck stamping your feet in anger in a court room).
Which will no doubt make for some fun court cases but will also take many many years to come to any kind of conclusion. I would recommend not to get your hopes up on that. Historically, copyright law has not been updated a lot to deal with any kind of technical change. And that's just inside the US. The world is bigger than that of course. A safe assumption is that judges will seek to interpret any new cases under centuries of existing interpretations. Because that's what they do. Any changes to the law would have to go through politicians. And then the interpretation of that relative to other laws is again up to judges. Which is why we have such wonderful things as DMCA and a few other attempts to close loopholes in copright law.
Of course time is not your friend here. By the time this rolls through the courts some years down the line, existing practice will have evolved to be hopelessly dependent on language models for just about anything. Including interpreting the law (e.g. chat gpt seems to be acing exams) and doing all sorts of complicated things in science, engineering and technology. So, any legal outcome that that is somehow illegal is going to be combination of unpopular, economically damaging, and therefore unlikely. Think big corporations ki
Ok, not a lot, but it has been updated. Chip mask is a good example: https://en.wikipedia.org/wiki/Semiconductor_Chip_Protection_.... If language model gets as important as chip mask, I don't see why similar update wouldn't be made.
Most of the people complaining seem to be a narrow subset of open source developers that like open source but not people doing anything for profit with their source code. I.e. not your typical multi billion dollar companies that are worried about their IP getting stolen. So, not a lot of lobbying power or ability to get organized.
If it is that simple then clearly Copilot need to include the licence of the code when they republish it. Changing GPL'ed code doesn't make it not-GPL code at any point. If it started out as GPL it is still GPL later in the line when a user sees it in Copilot.
See, if a comment like that crops up on a civilised society like this, it demonstrates that the barren wastelands of “the outside world” will have similar examples.
Did you write the absolute best implementation of X? I want to see it everywhere. Everywhere. In every single place where X is needed or discussed. Where all fine X are sold. Don't you? Or do you genuinely want to narrow the people who see your impl by some signifnicant percentage because you got frustrated with the ugly capitalism of a recent distribution mechanism?
If I invented Golden rice[0] I'd like to think I'd allow it to be sold at Panda Express and all the other evil capitalist chains, not just the local co-op whose business practices I prefer.
0. https://en.wikipedia.org/wiki/Golden_rice
This may seem an extreme comparison, but to one who deeply cares about Free Software as defined by the FSF, proprietary software is bad. That's the difference between people who prefer Apache/BSD/MIT licenses versus those who prefer the (A)GPL.
Fighting fire with fire is one option. Fighting fire with water is another.
Ideally, github should check the license of the code it's using to feed copilot, and only use code with permissive licenses.
Fundamentally it seems to me that Copilot truly does synthesise code from some store of knowledge (even if it's hard to understand what this store of knowledge is), and the problem is that it's synthesising code identical to existing code. There are legal tools and also rhetoric that are designed for dealing with this problem of "synthesising something that is identical to an existing thing", and it's different tools and rhetoric from the ones we have for dealing with the problem of "stealing or copying existing things". It's valid to have an issue with with Copilot ingesting your code, but unfortunately people are largely using tools and rhetoric from the latter category to approach the issue, and that slight misapplication is causing their issues to fall on deaf ears a lot of the time.
I expect that the copyright holders could reach to me, tell me that my code is identical to theirs, believe or not that I independently created the same code, and at least ask me to stop distributing it.
Of course
is easier to defend than because the chances that I didn't copy their code get slimmer and slimmer as the names of the identifiers and the structure of the code get more complex.If I actually looked at their code, it would be wiser from me to at least change the names, maybe also a little bit the structure of the code. That's the difference between being inspired by something and copying it.
TL;DR: if Copilot generates a copy, it's a copy.
Then you're in the clear, though you may need to convince a court that that's what happened. (Patents and trademarks could still be issues, but there's no copyright issue)
It's a common mistake because we are not used to LLMs.
If the LLM reproduces a significant portion of a program token-by-token, is it a derivative work and is it not fair use?
A derivative work is not fair use. If you end up with a significant portion of another person's program in yours, (such that a substantial portion of your program is in some way related to their program), that will likely be a derivative work - but the definition of a derivative work depends on your jurisdiction and use-case. If you're unlawfully using the source material to produce a derivative work, you cannot copyright that derivative work under 17 U.S.C. § 103(a), and under the same section, you can only copyright your modifications, not the original.
It would be hard to argue fair use in this case; fair use only really applies for parodies/criticism, reporting, and scholarly works - and generally that's an affirmative defense, rather than an express or implied right you have.
Honestly, Copilot is difficult because Copilot can't be the author of the code; the person who used Copilot is the "author" of the code, and I think they'd be the ones liable for copyright infringement if copyrighted content ends up in their code.
To argue someone performed copyright infringement, all you need is to prove (1) a valid copyright exists; (2) that the person had access to the work; (3) the person had the opportunity to steal the work; and (4) that protected elements of the work had been copied (afaik generally under a "substantially similar" standard). Copilot offers an easy way to check both (2) and (3) - a copyright holder could argue that people had access to their code through Copilot, and that Copilot offered an opportunity to steal the work.
I think of it the other way around: I'd be more trusting that LLMs aren't a problem in that sense, if the big commercial entities linked to them used their own code to train them as well as public repositories.
MS's codebase must contain much good training material, unless they think their own code is crap and are embarrassed to let the AI look at it!
I am still undecided, but erring in the direction of not wanting my stuff to be used to train them, though I've long since passed Douglas Adam's "35 year's old" tech barrier so maybe I'm just not liking change…
For now at least I won't be putting my own code in services like GitHub, then at least if someone else does or an LLM is trained on public sites generally, I've not explicitly agreed to anything that says the company can use my code that way - which you do (they say, I believe there is at least one court case brewing on the matter) when you sign up to the services (agreeing to their terms in the process) and use them that way.
Having worked at Microsoft, I'm not sure I want my coding assistant trained on Microsoft's codebase.
> legal tools and also rhetoric that are designed for dealing with this problem of "synthesising something that is identical to an existing thing"
I feel like I’m reading the exact same sentence that someone posted about if StableDiffusion included images or was just “learning patterns” and totally generative from 6 to 8 months ago.
People argued, oh so hard about it.
“But stable diffusion is GB in size, how can it have embedded full images of a dataset that is so much larger? It’s not possible.”
…but, it turns out it is possible (1) and it does, in fact, have embedded full copies of the source images.
Now. Let’s talk about code…
> and the problem is that it's synthesising code identical to existing code
There’s a word for that. It’s called copying.
Copying fragments, copying full text. It’s a black box, it spits out content that is indistinguishable (again, see stable diffusion, a lossy jpg of an image is the same image even when it is not bit-for-bit identical) from the input.
That’s copying.
The black box might be a neutral network, or a python script that reads a file from disk. The process is irrelevant.
If you memorise a block of code and type it out by hand with no reference, you are copying it.
These models copy code.
These models have embedded full text copies of some code.
What we do with that is an ethical question, but that it is true is not in dispute. It is true. It’s documented.
If you think these models are not copying at least some training data as output you are factually, and provably incorrect.
(1) - https://arstechnica.com/information-technology/2023/02/resea...
Black box. Input. Output. Is the output the same as some training input?
It’s not rocket science.
“…but humans…” argument is not relevant. This is not a human.
It doesn’t matter if there is a human doing it or not.
For your supposition to work, the input to the box would be only an abstract summary of the logical steps, and the output an exact copy of some other thing that was never an input.
In that case, yes, it would not be copying.
..but, is that the case? Hm? Is that what we care about? Is it possible to randomly generate the exact sequence of text with no matching training input? Along with the comments?
It seems fabulously unlikely, the point of being totally absurd.
Machines don't. It doesn't matter how fancy they are. The law doesn't care.
So yes, a human in a black box is different from a machine in a black box until laws change.
Wrong. Literally the whole thing about law - and especially about intellectual property laws - is that process is as much, if not more relevant, than the outcome. This is why "code as law" efforts are plain suicidal. This is why you can't just print out a hex dump of a pirated MP3 file and claim it's not copyright violation because it's just a long number that your RNG spit out - it would've been a good argument if your RNG actually did that, but it didn't, and that's what matters.
This is what it means when we say that, for just about everyone except computer scientists, bits have colour[0]. Lawyers and regulators - and even ordinary people - track provenance of things; how something came to be matters as much, and often more, than what the thing itself is.
This is what makes generative AI case interesting. They're right there in the middle between two extremes: machines xeroxing their inputs, and humans engaging in creative work. We call the former "copying", and the latter "innovation" or "invention" or "creativity". The two have vastly different legal implications. Generative AI is forcing us to formalize what actually makes them different, as until now, we didn't have a clear answer, because we didn't need one.
--
[0] - https://ansuz.sooke.bc.ca/entry/23
The court did not agree. They looked instead towards an anti-biker gang law that illustrated that a biker bar can be found guilty of assisting with gang crime, even if no specific crime can be directly associated with the bar.
The defense team argument - that prosecutors need to prove that a crime had occurred - failed. The courts only require that the opposite is not believable, which given all the facts around the case was deemed sufficient. In that question the process doesn't matter. If the court do not think it believable that copying has not occurred, any argument about "machines xeroxing their inputs and humans engaging in creative work" will be ignored.
They don't have the same rights to register a copyright.
Are you trying to make money off of it? Is it for personal use? Was it protected or public domain?
It just all depends on the circumstances. Human or not.
If you cover a song on YouTube, you can apparently be demonetised or attract copyright strikes from the original artist. But in software, you tell me an algorithm and I code it up, that’s an original work.
The lines all seem pretty arbitrary to me.
https://www.nationalgeographic.com/science/article/autism-ar...
Stephen Wiltshire is cool.
Any company that is using copilot or similar is walking right in to a massive legal minefield. Any engineer that chooses to use Copilot without explicit approval from the company should seriously worry about what target they're putting on their back. You don't want to be putting yourself in a position where your employer can say "They did this without our approval" or worse "Despite explicitly having told them not to", should there end up being legal action. Someone is going to establish precedence one way or another at some point, don't let it be you.
You mean, like the example the article used?
https://web.archive.org/web/20221017081115/https://nitter.ne...
"sparse matrix transpose, cs_"
Generated the authors matrixes code.
https://codeium.com/blog/copilot-trains-on-gpl-codeium-does-...
It produces an almost exact copy of some training data.
> These models copy code.
These assertions have very little weight - what matters is what the law says. And for now, in the jurisdictions that matter, the law is still silent. Maybe it will catch up via either legislation or case law, or maybe it won't. But for now there is no legal consensus on what these new things are doing.
Of course there is a moral/ethical component to this, and individuals are obviously free to hold personal views. But unless some global working consensus forms (unlikely) then I don't see how change will come from that.
I suspect that it's going to be difficult for politicians to find the will to do something about this issue, and then to get their heads around it enough to make good laws.
The legality aspect that you are injecting into the discussion is irrelevant
This is possible. For example, many functions f(0) return 0; if we can derive a function g(x) that generates y without access to T (ie. the training data) then we can plausibly assume that f(x, T) might indeed not be copying from T.
…but can we do so?
If so, we must admit that it is possible that f(x, T) = g(x) and no copying is taking place.
So, in the past 50 years of prior work can we come up with an example of a function g(x) that generates the exact code we see coming out of these LLMs?
I’ll grant, it’s not impossible.
For example, if stable diffusion generated sin wave patterns or fractals that exhibited “deep complex structure” and it happens that some artist had written code to do that exact thing, there would (I believe) be a fair case that stable diffusion was not copying the artist, it was simply a parallel implementation of the same generative code.
However, now my scepticism kicks in.
For code written by hand, that was not generated, we are suggesting that a LLM is a parallel equivalent implementation of a human mind, and it can, with purely “learnt patterns” replicate not only the intent and structure of known code, but the exact code itself, repeatedly.
I’ll grant. It’s not impossible, and it’s very difficult to prove, but it seems fabulously, unbelievably, extraordinarily unlikely that a clean room reimplementation would be identical, repeatedly.
We’re entering into the domain of assigning probability and then doing a zero knowledge proof here, but the solution is easy.
Just prove by counter example.
Train a model that generates such code without the code as training data.
You have now won the argument.
…on the bright side, you’ve also solved the 10 million dollar question of “how do I train my LLM when I don’t have enough training data” (because that’s what you just did; train it to do something without training data), so you’re now rich and give zero ducks what I think. Congrats.
> For example, if stable diffusion generated sin wave patterns or fractals that exhibited “deep complex structure” and it happens that some artist had written code to do that exact thing, there would (I believe) be a fair case that stable diffusion was not copying the artist, it was simply a parallel implementation of the same generative code.
I understand your pseudo equations and I understand [some] engineering.
Why are artists writing code? Is that to show the parallel of if artists did write code?
Unless artists do really write code but that’s beyond the scope of what I’m asking.
And that in that case, g(x) that can produce a perfect copy of the artist's work without having the image in the training set provably exists, and is the actual mathematical fractal function itself.
Basically if you take something then feed new info into it is that new thing a new thing or a derivative of that work or other? With these models the data and mathmatical spline is so smeared across thousands of endpoints it is hard to tell. However, I do not think the courts have to think about the details of how the models work. They can do something else so a jury can get its head around it.
But basically the courts will probably simplify it into copyright things go in one side. Other things come out the other that sort of kind of resemble the original thing. Do not worry about the details of how it is done. Is that new thing owned by the original copyright holder? Or many holders, as it is smeared into a hundred other items that may or may not have copyright holders? Or is it a new work? Or is it a derivative work just colored?
This could go either way. As someone put it very nicely here a few weeks ago. These AIs are like the most amazing auto complete you have ever seen. Now in copyright if you make something and I independently come up with the same exact thing. Also I can prove that I am in very little trouble. Now in this case without that input code that AI model probably would not predict that exact string. But it in some cases exactly predict it again but can predict thousands of other things also. Is that prediction copyrightable? What if it predicts part of the code but with something else? I as a 3rd party who did not create the model and just used it and got the code what are my liabilities? Then if it is what are the consequences of that? The courts will have to decide eventually.
I believe China is trying to limit AI powers but who knows how that will go.
> …but, it turns out it is possible (1) and it does, in fact, have embedded full copies of the source images.
The _not possible_ assertion is in response to people arguing that generative image models literally copy and thus "steal" the images in their training set or that such generative models "work" by infringing on said images. The assertion is not that the models cannot possibly contain _any_ copies of training data, the assertion is that the models are generalizing and can not contain a majority of the training data.
Now, let's look at some lines from your linked article titled "Paper: Stable Diffusion 'memorizes' some images, sparking privacy concerns", which happens to lead with scare quotes around "memorizes" and already hedges with "some":
> Researchers only extracted 94 direct matches and 109 perceptual near-matches out of 350,000 high-probability-of-memorization images they tested . . . resulting in a roughly 0.03 percent memorization rate in this particular scenario.
I do not consider a 0.03% rate of full copies particularly damning. Instead, it sounds like a flaw in training which could be addressed, which the article does in fact also mention being as a matter of overfitting to images that are over-represented in the training data.
> the 160 million-image training dataset is many orders of magnitude larger than the 2GB Stable Diffusion AI model. That means any memorization that exists in the model is small, rare, and very difficult to accidentally extract.
Which side's claims do you feel this line supports?
Generative text models should, in theory, also be generalizing and not memorizing. In practice, however, it does appear far easier to get Copilot and the like to spit out exact copies of code, complete with license text. Presumably, there's just not enough code out there to sufficiently generalize from, so we're seeing overfitting as a matter of course.
Or hell, the extracted potentially infringing and over-represented material might all be pop music set to variations of Pachelbel's Canon, and I'd pay to see that lawsuit.
1. The copyright on the composition. This can also include arrangements - for instance, Gershwin's original piano version of Rhapsody in Blue is now public domain, but the orchestral version everyone knows,
2. The copyright on the sheet music (the actual layout, spacing, editorial notes, things like that.. it's actually an insanely deep subject. I've got an 800 page book on the subject - which is referred to as music engraving, as up until about 40 years ago it was literally done by engraving the plates by hand. Much much harder problem than doing normal book-style text layout, as it's fully 2D, whereas text is basically 1d with occasional special cases. (NB: This copyright is really only relevant to the musicians, conductors, etc, but it does matter.
3. The copyright of the particular recording. This is the really relevant one. A 5 year old recording of a 500 year old work is very much under copyright.
No one sued Napster because their guitar tabs were being shared.
From GP: " they may largely consist of performances of public domain compositions."
My entire point is that the composition being PD does not mean RECORDINGS of it are PD.
few images of of millions
If Edison breaks into Tesla’s house at night, takes* the invention off his workbench, goes to his factory and uses it as a reference to build and sell the same thing, then Edison has stolen and copied Tesla’s invention.
In both cases we can say “stolen and copied”, but actual sequence of events and the legal recourse that Tesla has is different in each case.
This difference matters because open source is like Tesla having his workbench in a public place where people are encouraged to watch him work, play around with the inventions, help him build, etc. In the “Tesla’s public workbench” situation, Tesla’s recourse is debatably impacted in the first scenario, but it is definitely unchanged in the second scenario. Arguing as though Copilot is doing “second scenario stealing and copying” rather than “first scenario stealing and copying” sidesteps that debate about whether open sourcing your code made it ‘up for grabs’ for natural and/or artificial intelligences to learn from and/or memorize. Now, it’s not clear what side will win that debate - I personally think “this has infringed intellectual property rights” has a decent chance of being the victor. But sidestepping that debate is setting off peoples’ “you’re trying to pull a fast one” alarm and making them unsympathetic. That’s what I’m getting at here.
*: to avoid “steal vs copy depriving of property”-type discussions, postulate that Edison returns the invention to the workbench before Tesla notices
As to the linked article, I would quote from the article itself: “[The paper] is dense with nuance that could potentially be molded to fit a particular narrative”. I could say more: both the article and the paper state there were zero byte-identical matches, and 203 direct or perceptual near matches out of 175 million generations (or out 160 million training images) is 0.0001%, which argues against this very strong claim of “factually and provably incorrect”. But I think the argument I will actually make is this: I said Copilot was synthesizing identical code from its knowledge base, which is completely compatible with the claim that it is outputting code from memory (in the human sense of memory). What it is not compatible with is the claim that it is outputting code from memory (in the computer sense of memory).
AI instead has looked at all the existing code and it has learnt to do no more than what the existing code can do.
Do you see a difference?
Humans make sense of the world with an extremely limited amount of information. I have not read all of GitHub. Comparatively, I have not read a fraction of it. And I could not read all of GitHub even if I wanted to.
However, I do not need to read all of GitHub anyway because, as a human, I am capable of understanding.
Current generation "AI" is not capable of understanding. Current generation "AI" merely computes the probability of any given word appearing next in this sentence based on an utterly absurd amount of raw data.
That is not what humans do at all. We do not need this information. We cannot even process it.
> Do you see a difference?
Put another way: Some Markov models are Turing complete, so "merely" computing probabilities can be Turing complete with only minor steps, so trying to downplay the potential capabilities of models like this by handwaving about things we don't know is foolish.
We don't need that scale of input, but we also don't know if LLMs need that much input to do well, or if our current training protocols are simply poor. With ongoing work on reducing the training cost, it is at a minimum clear that current training methods are far from optimal.
Are you sure about that?
Yes I'm quite sure. Unless you're claiming that source code existed before humans?
It sounds like what you are saying is that the AI cannot write code which would do something it hasn’t seen in the training set. That is how I interpreted the “it has learnt to do no more than what the existing code can do”. Do i understand you right?
If so, how do you know that? Are you talking about the limitations of a particular implementation or a limitation of all AIs as a concept?
No, the problem is that it's illegal to do so.
Copyright law applies in both cases. If you create work that’s substantially similar to an existing one, you’re risking copyright infringement even if none of the original work’s content was reproduced in the mechanical sense.
If this were not the case, why would cover bands pay the original artists for rights to the songs?
“I learned to play ‘Yesterday’ by heart” doesn’t mean you can do anything with the song without paying the Beatles. The same applies to machine learning, if all the model has learned is to imitate copyrighted works.
If it was possible to point where the pixel values are actually stored in a jpeg file (i.e. run dd on the file and it spits out the pixel values), I would be a lot more sympathetic to the view that LLMs are not stealing code.
If you decode the jpeg file you do get the pixel values.
One could describe llms similarly.
I don't really care whether there's a lossless, lossy, or AI compression of my stuff. I care that it's my stuff.
If you don't want your code public, don't make it so. Because licenses are a thing of trust and you shouldn't trust anybody.
1. https://www.theguardian.com/commentisfree/2023/may/08/ai-mac...
I don't know if it's laughable, or utterly terrifying giving such a powerful tool into the hands of reckless wannabe entrepreneurs.
Weasel wording to evade legislation. It's not an unlicensed taxi, it's ride sharing. It's not an illegal hotel, it's couch surfing. It's not code licence infringement because it's learning.
It's lawyers finding loopholes for finance to avoid expenses that gives them an edge over the poor suckers who play by the rules. The tech is just a tool to this purpose.
And most people forget TANSTAAFL. The costs for the cars and their infrastructure, the load tourists put on a place, and the effort for writing the code are still there. The "innovators" just found a way to make somebody else pay for what should be their cost center.
I don't see the point of this because quite frankly - if you want to prevent AIs using your data, you already lost that battle in the moment you uploaded your Code to the publicly accessible internet.