Ask HN: Why do devs feel CoPilot has stolen code but DALL-E is praised for art?

58 points by amrrs ↗ HN
Both DALL-E from OpenAI and Github CoPilot work on the same philosophy. While one receives a massive praise online the other is being criticised that it's working with copied code and just pastes copied code.

Why is this difference in opinion?

64 comments

[ 2.7 ms ] story [ 141 ms ] thread
Because good artists borrow, but great artists steal.
Perhaps one is an interesting novelty that while fun to tinker with, has little practical value. While the is potentially a tool.
Because, generally, art is not considered a real profession. "Making it big" in the art world is an incredible gamble, and even more-so today with so many attention-grabbing alternatives (movies, TV, YouTube, blogs, music, etc.) The impact of AI-generated art will barely be felt, in my opinion.

While software engineering, on the other hand, is considered a "real" profession. Millions of people make a steady living coding, designing, testing, architecting, etc. The impact of AI on the software development work-flow will be immense. Tooling has already impacted GIT PRs at my place of work, and when these tools get smarter the impact will only grow. And, if the pattern holds, such tooling will be introduced earlier and earlier in the design/development workflow.

I'm probably way off, but at the rate things are progressing, I'll give it 10-to-15 years before software development paradigms such as cucumber test files or hexagonal designs/drawing are directly transformed into coded, reviewed, tested, running and deployed solutions.

(comment deleted)
DALL-E is maybe copyright infringement, CoPilot is whitelabeling code sometimes in flagrant violation of their licensing terms, which may require such things as attribution, or even that the derived code is licensed in the same way.
Imagine being sued by Oracle, just because your cat sat on tab key, and auto completed the java compiler :p
Ooh, give a cat access to the tab key: behold, CatPilot.
Because Copilot can generate verbatim copies of code, with only the whitespace changed in some cases.

The AI art generators are trained on copyrighted work, which is still a legal grey area, but they don't stamp or copy/paste parts of the source images unchanged into larger ones. Though I'd imagine if you used the same algorithms for something like logo or typeface generation you'd run into trademark issues.

Except it now has a setting that will prevent output that matches against the training set, and even before that verbatim output was only in cases of blank files, particularly in blank projects.

This is a straw man with examples caused by going out of the way to create the environment in which they would occur, when in fact they are edge chases outside the vast majority of usage.

I'd be willing to bet there's a MUCH higher likelihood that a junior dev is copying in part of a SO answer without considering licence than that Copilot generates an identifiable and unique snippet in an actual codebase with thousands of existing LOC.

> even before that verbatim output was only in cases of blank files, particularly in blank projects.

Don't be so sure about that.

I don’t know how it is amongst FAANGs but I’m sure more than one of my previous employers chief legal counsel would have had an aneurysm if they saw how much code was copy and pasted from not just stackoverflow but devs asking their friends at other companies for help, who had no issue copy and pasting the code they wrote at work over.

I had somewhat of a legal background? Understanding? Whatever term means I’m not a lawyer started down pre law and learned a few tidbits so I was concerned enough to always start from scratch instead of copy and pasting when helping. Fuck me though if a good 25%-33% of my coworkers functions weren’t an exact copy of what’s in other companies codebase

False premise? Loads of artists hate DALL-E and AI art in general; they typically have much less feelings about Copilot. Unsurprisingly, for devs and DALL-E, it's the other way around.
Fully agree.

We've been in discussions with image agencies who are afraid that their images will be remixed by AI and then distributed as CC0. If that happens, they are competing against free "derived works" of their own photos, but with the AI remixing proving that in court will be difficult and expensive. The result is that prices will go so low that they can't pay their photographers a fair salary anymore.

I believe this is also why Adobe and some photography industry groups now heavily promote their "Content Authenticity" initiative which aims to make it illegal to use photos where there is no matching camera RAW file with Metadata. Because, obviously, AI-generated images don't have that.

i strongly feel that 'in the style of' prompts should be outright blacklisted honestly. other cases might be more vague but if you tell an ai to make a painting like james gurney, it can only do that by training itself on james gurney's body of work. by extension the creative commons license needs to be modified so that artists and photographers can decide whether or not to give permissions for their work to be datamined by ai.
I agree. I wonder if using "james gurney" as a description here is a trademark issue.

That said, I find things like "DALL-E 2 Variations" much more worrying. If you look at their flower shop example, it's difficult to argue that we still need stock photographers if you can get those variations for "free". But of course, the AI can only produce the variations, because some human took the original photo. So in effect the DALL-E variations then become a tool to obfuscate the original photographer so that you can use his/her work without paying.

As for the datamining, I believe that's unrelated to the license. I believe in the US you are legally allowed to train your AI on any data, no matter what the original license was. Otherwise, Co-Pilot on GPL or MIT code would also not be permitted.

The latter might be feasible (Though I doubt databrokers will let the current laissez-faire data-rights landscape go without a fight). The former is both wrong and infeasible.

Bigger models get better at copying style with fewer examples. At some point, it'll be enough for a big model to see sufficiently few examples of real people drawing, "X in the style of Y" to get a sense for how to draw a thing in the style of Y.

Or an end-user could just feed a single instance of parody art in some artists style, ask for variations, in-paint what they want, and voila! Copyright free [citation needed] style transfer.

It will always be possible to launder style, even while adhering to promises to never touch an artists work (directly). Training on an artists entire body of work is not the only way (and thanks to copyright, a not even desirable) way to do so. edit: Open source models will also mean blacklists are unworkable.

> if you tell an ai to make a painting like james gurney, it can only do that by training itself on james gurney's body of work.

How is that different from telling a person to do the same?

> Introducing DALL-E 3: Now with matching RAW files, soon available in a closed platform near you
The CAI idea is for camera manufacturers to cryptographically sign the RAW inside the camera to prevent that.
"Coming soon: DALL-E-in-a-box†".

† light-tight box contains high-resolution WiFi-enabled tablet hooked up to camera whose button is automatically pushed by a small motorized mechanism. Subscription required.

Add a Blockchain and that sounds oddly realistic.
Is photography becoming obsolete as a means of pure artistic expression?

It might stay useful for documenting reality, and in the space where art and documentation meets (e.g. it's still nice to have pretty pictures in your review of the latest smartphone, so that incorporates a manner of artistic expression).

Interesting to note the official position is that you cannot copyright AI generated art due to the lack of human authorship.

[1] https://www.smithsonianmag.com/smart-news/us-copyright-offic...

The area of AI and IP is going to be a hot topic in the next few years. There's already massive forces working on changing that stance for everything from copyright to patents.
You say that, but the two best-known generators (DALL-E and Craiyon/dall-e mini) both claim ownership over generated images. E.g. they both claim that generated images may only be used non-commercially, and that commercial use is restricted or not allowed.
They (probably) do not own the copyright to the generated images, as they did not have any creative input into the process. However, they can ban you from their service, as part of their terms of service, if you do something with the images that they don’t like.
I suppose the legal specifics are uncharted territory, but at least with DALL-E the operator explicitly claims ownership of all generated images.

> 2. OpenAI owns generations created by DALL·E; you own your uploaded images.

I suppose that could refer to ownership in some other sense besides copyright, though I don't know what that would mean.

It’s uncharted territory but only somewhat, as it is generally accepted that the creators of a tool do not have any copyright claim to the works that users create with that tool.

e.g. Adobe does not own the pictures you create with Photoshop or Illustrator. Even if Photoshop is filling in the pixels with content aware fill, Adobe doesn’t get any authorship rights to your works.

As far as copyright is concerned, the author is the human in the loop making the creative decisions. For generated images, that is primarily the person writing the prompt, and secondarily the creators of the training set (the extent to which the authors of the training set hold copyright to derived works is a more interesting legal question).

OpenAI are neither writing the prompt, nor did they create the images in the training set. So it I don’t see where their authorship claim is supposed to come from.

They could ask for copyright assignment. I haven’t read the full terms of service so I don’t know if they require copyright assignment. But them merely asserting that they own the images is not a copyright assignment, “We own the images” is legally meaningless by itself - own what form of IP, through what mechanism?

It could be argued either way, since in this case the tool does a lot more more than a content aware fill.

If I were to ask you to draw a rabbit, you'd own the copyright on the picture despite me giving you the prompt.

In the case where a neural net is given the prompt, who owns the copyright? The prompter, the neural net, or the creator of the neural net?

Commonly, the for-hire aspects of the copyright laws would come into play. If you have a prestanding relationship, and receive a benefit from the prompter, then the prompter is often the owner of the copyright in the US, despite not being the creator.

A neural net's existence might be considered as a benefit in that case, so you don't need to dive into possible personship before establishing the owner.

> I suppose the legal specifics are uncharted territory

Depends entirely on where you live. [0] Whilst the attached case was rejected in the UK and US, Australia reached the conclusion that AI itself, not its creators, might have ownership over its production, as... Insane... As that is.

(There's another High Court case to come. Hence the 'may' have ownership.)

[0] https://www.judgments.fedcourt.gov.au/judgments/Judgments/fc...

Code is shared with explicit licensing. Art exists in a more vague space, where the line between stealing and influence is subjective.

If your code is released with no license, it should be fair to assume free use. But if it is licensed, tools like Copilot should be forced to respect that.

If code is released with no license, the correct assumption is you should not do anything with it, in my opinion.
Are you allowed to look at it?

Yes.

Are you allowed to remember it?

Yes.

Are you allowed to use similar patterns / naming conventions / algorithms when your are trying to solve your own unique problems?

…I hope so, or you won’t be much value to your employer.

That’s all an AI does. Same thing as you, it’s just more efficient (in some ways).

> If your code is released with no license, it should be fair to assume free use.

Legally? The opposite is true. Source-available is not open source, and is emphatically not public domain.

There's a reason that StackOverflow and all the other code bins have legal agreements that you are notified of when using them.

If I can give a different view to everyone else: because it’s much easier to demonstrate technically. If copilot writes some code that’s copyrighted or semantically identical to another, it’s easy to copy the text and search Google for it. Bam, copyright infringement.

For DALL-E, I can’t select components in it and search for it. It may well copy components on other peoples art verbatim (is that the right word for images?) or exactly, but I can’t show where it’s ripped it from.

The real reason is because this time it hits developers’ pockets.

You saw the same thing with how online forums which were predominantly dev/tech focused (/., Digg) used to decry any attempts to protect online music/video, but would simultaneously complain about software piracy.

It boils down to hypocrisy, really.

Personally, I believe the underlying difference is that the parent company of DALL-E is a nonprofit with clear cost recoups for their api pricing and the access to DALL-E is currently free. CoPilot is from a for profit company which has been around enough for a lot of people to have strong opinions of. CoPilot was initially introduced as a product that would have a subscription or license cost. There are copyright and attribution differences and some similarities but from someone on the edge of the development realm my reaction was firmly because of the free/at cost transparency for OpenAI versus the clear 'this is a product to make money from'. I'm not saying that the latter is an inherently bad thing but it absolutely colors the way you feel about it.
Yeah but Copilot is actually generally available. DALLE 2 is invite only.
I thought marcan on Twitter had a pretty good take on this:

> Source code [is] relatively low entropy compared to, say, images. It's a lot easier to see how feeding a pile of photographs into an AI that maps them to subject erases the copyright of the photographers, because the subject isn't copyrighted.

> It's trickier when you feed artwork or music into an AI, because the AI might reproduce specific artistic or musical choices that happen to pinpoint a copyrighted work too well. With prose it's even worse. And code has to compile, [so] it's even more constrained!

https://twitter.com/marcan42/status/1539825034860335105

A small change in art is considered materially different, isn’t it? We don’t charge artists for making similar art inspired by others.

Combine that with rulings in other areas of IP law, such as the patent office ruling that an inventor must be a human and AI cannot be assigned IP rights[1], I think that would lead towards DALL-E created art being assigned to the human who prompted the tool for art or the creator of the tool.

Compare this to software when you can be sued for using the same pieces of code, and that seems to be the root cause for the difference

[1] https://www.ipwatchdog.com/2020/07/13/artificial-intelligenc...

I don't consider Copilot to be stealing code.
I (foolishly) hope that this will lead to a collapse in copyright.
I think what is more likely to cause that is when someone does this for music, as the RIAA is 100% going to sue whomever pulls off "give me a song about my ex girlfriend that sounds like Taylor Swift singing a cover of a song in the style of one from Bob Dylan".
I find it weird people are focusing so much on the tech, instead of the kind of person who would use it.

After decades of professional software development, it should be clear that code is a liability. The more you have, the worse things get. A tool that makes it easy to crank out a ton of it, is exactly the opposite of what we need.

If a coworker uses it, I will consider it an admission of incompetence. Simple as that.

I take it you write all of your code in notepad with no linting, formatting, or any of the other cheater plugins that incompetent devs use?
I don't use autoformat, because it gets things wrong constantly. E.g. taking two similar lines and wrapping one but not the other, because of 1 character length difference. Instead I explicitly line my code out by hand to emphasize structure.

I also hate 90% of default linter rules because they are pointless busywork designed to catch noob mistakes.

These tools keep devs stuck in local maxima of mediocrity. It's like writing prose with a thesaurus on, and accepting every single suggestion blindly.

I coded for 20 years without them, why would I need them now? If you can't even fathom coding without these crutches, and think this is somehow equivalent to coding in a bare notepad, you are proving my point.

This is an idiotic take. Tools that increase efficiency are good actually
It is not more efficient to write endless permutations of the same concepts. It creates a maintenance nightmare.
because Copilot was made by Microsoft and DALL-E was made by Google.

that's literally all it is.

Except it didnt. Dall-e and copilot are both microsoft because microsoft bought openai.

Google created imagen which is better than dall-e.

Ask some artists, my art collective is using Midjourney as part of a project, and we are actively discussing how to deal with the subject of authorship and credit.
because dall-e is mostly a fun novelty, but if you were to use more advanced versions of it for actual image generation, i would definitely be concerned how much of someone elses image ends up in the thing you generated.
Artists who end up seeing parts of their artwork on T-shirts getting a “no, but an AI did this” will be equally angry.
I can also see it going in a myriad of other directions, though my biases are clear.

- People at art conventions walking up to artists and asking them what model and keywords they used, insinuating that no human talent was involved in the process.

- An unresolvable divide between "AI is just another tool" and "AI instills distrust" spreading to mainstream discourse.

- Comments on art posted online filled with requests for "keywords please," because that's what AI would train us to expect out of creators.

- Commercialization and legitimation of plagiarism through the "just another tool" argument.

- No longer being able to detach yourself from the AI debate every time you see a photo or illustration online. No longer being capable of knowing if there was a human behind a creation. Constantly having to worry if the person you're chatting with is the entire human, or if some parts were outsourced.

- Never being able to go back to now, before the sphere of illustration has been "polluted" from the proliferation of AI art, when we cannot uninvent the tools and anyone can spin up another private instance. Banning the use of a technology is impossible but perhaps nothing less would prevent such a hypothetical outcome in the face of perceived progress.

Copilot is just showing a problem we had all along more clearly.

Commercial entities take things that are written and maintained by volunteers in their free time and barely give back. And this culture has just developed in a way that this is at most a moral failure, not a legal.

People put it on the internet with a permissive license because they want others use it. This is great because people love sharing, but now, before you know it, you're suddenly supporting large enerprises and not just people that want to put cool things together like you. And you don't get any kind of compensation, even though it'd be peanuts for the company to pay for the time you put into maintenance and new development.

And if you stop, someone else will always fill the void, eager to work for free.

Maintainer burnout is a real thing and I wish the largest user of FOSS code would do their part to make the time spent on every piece of code they use worth it.

Half of "open source" is a success story, the other half is just sad.

Copilot just automated this entire process and sells the product of code back to developers, not enterprises.

One distinction is that CoPilot is only trained on open source code. It transfers value from the most self-less people to profit oriented people. No such distinction exists for art.

Actually I think it should be up to the art community to decide how they want to deal with dall-e. I know that musicians have a very complex system for dealing with samples, and arrangements and lyrics. As an outsider I have barely scratched the surface on how it works, so trying to understand the why is hopeless.

Dall-E is just as much enclosure and theft of the commons as copilot is.

Main difference is there isn't as hard a line between it and lower resource projects such as disco diffusion.