Whistleblower mysteriously found dead after exposing industrial scale Copyright abuse. Isomorphic plagiarism of $9Tn in FOSS and user content is the product. =3
> The New York Times article cites Stanford University law professor Mark Lemley, who disagreed that generative AI services violate copyright law, and intellectual property attorney Bradley Hulbert, who said a new law might be necessary to settle the question of legality.
> Months after Balaji's death, which attracted significant public attention, Hulbert told Fortune magazine that Balaji's essay "[reads like] the argument of a really smart non-lawyer who read up on the subject but does not have a thorough understanding".
If there's some kind of industrial-scale intimidation campaign that's stopping IP lawyers from litigating the case of their lifetime, that's an even bigger story than OpenAI taking out a hit on somebody. It seems like they're agreeing that the copyright abuse was never hidden, and it's sufficiently transformative enough that nobody could argue it's illegal.
Whatever the original intention, this is clearly a bug and should be fixed. But should ChatGPT sign its cartoons with its own name or leave them unsigned?
No one should be allowed to claim they drew something when they didn’t draw any part of it and LLM’s are not people/can’t work without a person. We don’t credit pens and paintbrushes after all.
One could argue nobody should be allowed to claim it. It just exists.
Even if the person can't claim the copyright of the image produced they ARE responsible for the use of their tools and what they do with the output.
In this case, they released an image with someone else's signature on it. That is wrong, the person should take the blame for that.
The person releasing the image may take it up with the AI service that their tooling led them into making such a mistake. But good luck with that in court...
i’m not saying it’s airtight, this is a pretty unbaked idea. But I think it pretty clearly has some legs to stand on. Can’t be any worse of an argument than somebody prompting an AI for a (facsimile of a) photograph and going “I made a photo.” And mind you I’ve heard people argue that that absolutely constitutes taking a photograph (which I wholly disagree with) here on HN.
Somebody “made something.” But just because you do something doesn’t mean you get to claim whole ownership of it and get to sign it with your name. Plenty of examples in life.
I have no problem with people using LLM tooling. It's the bit that happens after that's the issue.
LLM generated picture with artists signature. That's no problem.
Person takes this picture and proceeds to share it around is the problem.
What annoys me is when "LLM hacks website!". Uh, it just went and did that completely unprompted? Nope.
Somebody ran it on a computer somewhere, gave the LLM some kind of prompt+access combination - however precise or vague - and THEN the LLM did a thing.
Power tools are well known for making human bits go missing. But nobody says "The angle grinder did it on it's own! Nobody is at fault!"
Maybe the person wasn't trained. Maybe it's the wrong tool for the job. Maybe the manufacturer of the tool is at fault for selling a product without appropriate safety guards. But the LLMs aren't doing self sustaining thinking for themselves just yet.
> Person takes this picture and proceeds to share it around is the problem.
It’s funny I talk about this a lot. I haven’t quite worked out my metaphor, but it feels kind of akin to somebody writing a poem for the first time or drafting a short story. They drink a bunch of coffee, hammer it out through the middle of the night, then blast it out to the world expecting praise and adoration (whether they’ve written before or not). But then they get upset when people say it’s not complete or it looks rushed. Indignant even.
Despite how quickly LLMs turn out things and how little effort you can put in to get something out of them, people seem to have this weird cognitive dissonance about it. They feel like you are attacking them and then passionately defend their tools. But they didn’t really put that much effort in, so why are they bothered when you treat it that way?
I think what you're seeing is the probability of a particular signature or style of signature appearing on a particular style of cartoon, not an intent to sign.
This. It's also while you'll sometimes get a mangled Getty Images watermark on some image generations, or a logo in the bottom left corner. If it's a prominent feature in the training dataset it'll show up, exactly how these models are supposed to work.
The 'bug' here is whatever post-processing step or system prompt is in place to steer the model away from doing this.
For the same reason I'm allowed to draw, paint, or write things very similar to what others have drawn, painted, or written but I have to sign my own name not theirs.
I don't even believe too much in Intellectual Property but the signature amounts to a false representation, rather than a resemblance. The pixels are not special in the image file, but they're special when we look at them. This behavior should be trained out of the model.
I follow anti-LLM discourse quite a lot, and across the main bulletpoints: energy/carbon emissions, content worker harm, job displacement, deskilling, mental health effects, and copyright/plagiarism, the plagiarism one seems to have the most attention, and it's also the most solvable, if there were only more serious effort on ethically sourced models that can actually do the real science / math / code work that is what LLMs are best at. The whole world of LLMs to create videos/books/art/literature is where most of the offense is (the video/imagery side of it is where most of the energy/carbon emissions problems are too. and content worker harm).
I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.
i think math is close to art (just to be a contrarian, but kind of really)
its somewhat funny that math people are in a conundrum as to support or not support but this might partially be because some wish to believe that math itself is and can be useful and therefore accelerating is good
but the art people have no such delusions so they’re just strictly against
imo proof writing is more akin to art than coding/tech but…
Yep. I’d probably be a lot less chastised in some circles for using Claude Code at work if it wasn’t misconstrued as being in support of, I don’t know, encroaching on the hypothetical commissions of a chronically online instagram furry artist or something.
It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.
I am skeptical of there being sufficient data to build “ethical” training datasets, and I’m confident that much of the same contingent will (somewhat rightfully) argue that ‘second-generation’ copyrighted AI material has already irreversibly made its way into every modern dataset.
2. Acquire rights/licenses to any datasets that do not fit #1. e.g. the Google deal with Reddit for 60m/yr.
3. Offer programs to have creatives willingly submit their data, with some sort of residual output based on the number of times their assets are sampled.
4. If all that is still not enough, hire creatives to create assets for you. This is something Spotify did recently with "ghost artists"[0]. The intentions here are suspect, but a non-consumer facing artist providing work for an LLM wouldn't have the same ethical dilemmas
5. Lastly, if all that still isn't enough: governmental programs to either provide grants, subsidies, or more outreach to get the ball rolling.
Would this cost tens, hundreds of billions of dollars? Yes. But clearly, that was not a barrier to entry for the industry anyway. So we can chalk this down to the personality of leadership or the wider culture of modern big tech
I agree with most of your post, but I don't think I agree that method 2 (Reddit deal) is necessarily ethical. I know it's too high of a bar for our nation to ever clear, but I think explicit author opt-in is the only ethical source of AI training data.
Like, legally, I'm sure Reddit had the right to sell it, but probably over half their content was written before ChatGPT was ever announced. The TOS allowing reddit to make "derivative works" was largely understood to mean things like cropping photos, using your viral post in an ad, or maybe auto-translating your comment.
>“I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains”
Being able to prove such gains in better products would be a start. And an emphasis on how it assists existing engineers/mathmaticians/researchers, not that any accomplishment made with AI assistance is "AI solves problem".
I don't know whatever happened to "words are cheap". I guess it literally made money to say words, so that adage is false for the time being.
>I am skeptical of there being sufficient data to build “ethical” training datasets
Well if all those scam job ads paying 100/hr to create AI training content was not a scam and instead the approach from the start, there may have been a chance to bridge that gap ethically. The industry chose to break things and is trying to act mad that people are mad at all the broken stuff.
These results are entirely a consequences of the actions chosen. And I don't believe there was ever an honest consideration of there being ethical training datasets. They just thought they could brute force society with fearmongering and bribes. The BOTD was already low in the beginning but completely gone now.
I think you can train on math /science using synthetic generation to a significant extent. Training for coding requires more of the "scraping github / stackoverflow" angle but IMO that's a shallower hill to climb than scraping copyrighted art and literature.
There are actual models trained on ethical datasets but they are obviously not very high powered. If companies with the resources of an anthropic or openai were doing it (ha) it would be more feasible
> It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.
That’s not a justification. If a company were poisoning the water to your home as a byproduct, would you be satisfied if they told you “we don’t necessarily disagree with you about polluting the water, but in our field—which you do not understand, and in which the underlying build process is often not the water pollution—what we’re doing presents very real productivity gains”?
> I am skeptical of there being sufficient data to build “ethical” training datasets
Then you don’t build any. What fucked up world we live in where people think it’s OK to be unethical because they want something and can’t think of any other way to do it. What monumentally selfish rotten babies.
this is not whataboutism at all. This is my conjecture:
AI having negative externalities is not the sole deciding factor in deciding to eradicate it
then my argument: negative externalities are tolerated in many areas where we deem the topic is of enough value that the externalities are managed. there are many examples of things with negative externalities, much much larger than those of AI, that aren't being considered for eradication.
so your argument is implicitly, "AI is not worth these negative externalities"
which is a common opinion among people who dont use AI at all.
> If a company were poisoning the water to your home as a byproduct, would you be satisfied if they told you “we don’t necessarily disagree with you about polluting the water, but in our field—which you do not understand, and in which the underlying build process is often not the water pollution—what we’re doing presents very real productivity gains”?
no but I also wouldn't declare that whatever it is that company does should have its entire industry eradicated. Poor industrial practices can be mitigated while not abandoning the product being manufactured.
Looks like you have a very utilitarian approach to it, in that case, would you be fine with eradicating just diffusion and video models?
By your logic, they don't particularly contribute to scientific productivity/advances: it's not like you can use stable diffusion to generate an architectural diagram.
GenIA for images and videos is only really used to copy artstyles, make fake menus and create misinformation so wouldn't you agree that eradicating this part would not only preserve your described "useful applications" but also allow providers to allocate more resources to "useful" and "scientific" AI?
I know that local models have become decent enough that this isn't really doable, but I'm only proposing this thought experiment because you seem to think people are saying this should be all or nothing. But the truth is, if you get a genie to wipe away just diffusion models, the entire internet gets much better (or at least back to regular levels of bad), no one loses anything, and a whole lot of people stop complaining/campaigning against the productivity gains in your field.
A reasonable next move, if the US was interested in acting like a democracy, would be legislation forcing this decision:
- prove that you had the rights for all of your training data
- open source the model
Give the labs a 3 month grace period in which to comply, so competition can persist even with dubiously sourced data, but the people can't be locked away from derivatives of their contributions for any significant amount of time.
> I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.
I disagree that they can be separated. Practically, I think they can't. Because the mere invention of new tools inspires even more AI advancement and that in turn will cause the other side (artistic side) to degenerate even more.
I'm anti-LLM all the way, 100%, no exceptions. Zero tolerance.
So you recognize that discovery cannot be cut off from other discovery, that it's never just one or the other, and your solution is to say shut down all discovery?
you can distinguish between the LLMs you have zero tolerance for and a system like Google Translate? You have a sharp line you can draw for when something becomes "an LLM"?
I don't really see the difference, code is protected by copyright (or copyleft) as much as art is, and yet the LLM scrapers use it without scruples. Same goes for math and science publications.
Artists take pride in the intentionality of every brushstroke, and for them any given work is much more likely to reach a point where it's considered "finished".
While coders may care about the craft (and I do), it's not as if the value of my code is in the exact variable names I chose.
I dunno man, there’s plenty of art out there that’s just splatter art and spinning canvases and splashing paint all over them.
Meanwhile there’s endless discussions about the right way to name variables (not too shot, not too long, try to be self documenting, but not to the point of putting types in the name like we used to), and people definitely get judged by their variable names, it’s one of the first things someone will point out when they look at a codebase. “Why are all the variables a single letter this is bad code!”
openai and anthropic trained on actually stolen data since it was pirated datasets.
google OTOH already had a lot of this dataset in their possession (e.g. Google Books etc), still questionably licensed for how they used it, but not quite as bad. They did apparently break through NYT paywalls and stuff like that though, still theft.
While that is true, theoretically a regulation could be enacted that output tokens must focus on STEM research and other practical tasks and the LLM must refuse tasks outside of those areas, just as Claude disallowed cybersecurity tasks. Obviously this would never happen, but the theft of the training data wouldn’t matter as much if the usecases were less sinister.
In my experience, the science/math/code crowd don't care about copyright/plagiarism as much as the video/art/literature crowd, so the first crowd turns a blind eye to most of the latter crowd talks about.
Code can be art, and copyright/plagiarism is real. It sort of boils down to how much it bothers us.
I don't think it is likely that they could get enough data without stealing. It would be incredibly costly to have to pay artists to church out art just to train an AI.
This has been a perennial problem with my own generated comics with both Nano Banana Pro and ChatGPT (all generations). I often have to put in an extra edit to erase the false signature. It is annoying and I'm unsurprised most users don't bother.
Despite what all the clickwrap warnings and "AI can make mistakes" subtitles might lead you to believe, the service offering of AI is explicitly designed to be as "one and done" as possible. The inherent nature of these tools is to service laziness, and disincentivize too much scrutiny.
I am not reading that. XKCD, one of the most popular web comics of all time, is STILL stick figures. There's no excuse not to make the entire thing yours.
> I often have to put in an extra edit to erase the false signature.
Given that they can reliably do this, I'd think it should be trivial for the harness to automatically insert a "review the image for anything that looks like an artist's signature or other blatant indicator of plagiarism, and fix it" pass.
Yes, but that could double the cost by default! Which is probably why they don't. On reflection, coding bots do implement something like this with the verification tic they seem to have.
Engineering manager at my company put a comic at the end of our sprint demo that was signed bloper. Except it wasn’t funny at all, and kind of weird. I asked him, sure enough it was ChatGPT and he didn’t notice the signature.
This is completely silly. If you don’t think LLMs can reason, you’ve either never used them to do tasks that require reasoning, or you don’t understand enough to recognize what’s involved in the responses you get.
In this case it’s clearly the latter, because you’re confusing image generation models with LLMs. There are very big differences between the two. No-one is claiming that image generation models are capable of reasoning.
The thing about a generative language model that’s trained from a massive but unknown corpus is, it’s practically (if not theoretically) impossible to evaluate the extent to which data leakage contributes to any particular output.
But I would argue that, as things currently stand, “sophisticated engine for approximately querying a pastiche of the results of human reasoning that comprise its training corpus” remains a more parsimonious explanation than “it’s doing actual reasoning” for how this neural network architecture produces the phenomena we’ve been observing.
>sophisticated engine for approximately querying a pastiche of the results of human reasoning that comprise its training corpus
Well if the thing can find and fix bugs in something that is using non-mainstream stuff that is surely not in it's training dataset, that's better than a rubber duck already. Whether it has soul is a different question of course.
Don’t even bother. These people almost always have some goofy ass, non standard, fluid definition of “thinking” or “reasoning” that cannot ever be met.
My opinion of their reasoning capability is based in part on (proprietary, non-published, only internally peer reviewed) experiments on GPT-series models’ ability to perform a suite of formal and informal inference and deduction tasks.
Perhaps you could argue that “appropriately applies syllogism to arrive at correct conclusions” is too high a bar to set, but I don’t think it would be fair to call it a “goofy-ass”, “non-standard” or “fluid” element of a reasoning capacity assessment.
But you're not just saying they are insufficiently good at reasoning, you're saying they're (probably) not "doing actual reasoning". So we need to know how you are defining "actual reasoning".
I don't think the bar for an actual reasoner can possibly be 'always appropriately applies syllogism to arrive at correct conclusions', because in that case nobody in the world is an actual reasoner. And if your bar were 'sometimes appropriately applies syllogism to arrive at correct conclusions', it's hard to understand why the current generation of AIs doesn't meet it; they are clearly capable of doing so, at least to all outward appearances. (Maybe you think their apparently successful demonstrations of reasoning are illusions, but again, you would need to define what counts as "actual reasoning" vs. a superficially convincing simulation of it.)
Like I said, earlier, it’s proprietary work and I’m not at liberty to discuss in detail. So all I can really say in response to your specific criticism is that one throwaway example for the purpose of illustration is not the whole of the thing.
More generally, it’s important to keep the burden of proof in the right place. The idea that this particular neural network architecture is performing reasoning tasks is really quite extraordinary, and we should demand an unusually compelling case to be made before accepting it.
Compare to the example of Douglas Hofstadter and computer chess. In the late 70s he predicted that computers would need general intelligence to compete at the top levels in chess. When computers did start competing at that level, he did not automatically assume chess programs are intelligent. He took a look at how they are implemented, decided that the idea that that particular algorithm was doing abstract reasoning was implausible, and instead re-evaluated his assessment of what it takes to compete with a top tier human at chess.
I tend to align with Hofstadter’s approach to the question. Like I pointed at in a cousin comment, arguments that what we see represents reasoning tend to hinge on a common logical fallacy.
>More generally, it’s important to keep the burden of proof in the right place.
You have misunderstood the 'burden of proof' quite badly here. The extraordinary claim here is that what looks like a duck, quacks like a duck, acts like a duck is in fact not a duck even though you are simultaneously completely incapable of proving it is not a duck.
Just because you have placed humans on a pedestal does not mean you are not the one making the extraordinary claim. Hofstadter is entitled to his opinion and while you are free to follow his 'approach', you'd do well to consider that neither you nor Hofstadter in fact have any idea whatsoever how 'abstract reasoning' is implemented in even the systems you are certain posses it. Don't be confused that you are doing anything more than begging the question here.
Well, like I was hinting at above, it's actually based on a close analysis of all the ways it doesn't quack like a duck, together with some serious consideration of how plausible I believe it is that a decoder architecture would be capable of performing generic symbolic reasoning tasks. The crux of the problem is that general reasoning involves actively selecting and then manipulating a symbolic model at least semi-independently of language generation, and that's a very different shape of information flow from next token prediction, even with MHA.
I would certainly be open to hearing a detailed explanation of how that kind of thing might actually support reasoning, but, absent that, pointing out that I don't understand how brains do it (which is true) is the pot calling the kettle black.
I actually want to turn the tables on your claim about humans being on a pedestal, and this is another spot where I believe I side with Hofstadter. We humans have a tendency to believe that certain behaviors are so special that the only way to accomplish a similar outwardly-visible behavior is by using similar underlying mechanisms. I see a healthy measure of chauvinism in that assumption.
As I mentioned earlier, these LLMs are trained on vast collections of both the inputs and outputs of genuine human reasoning, and potentially have enough parameters to record a large proportion of that information in the form of probability distributions. And even something as crude as a singular value decomposition on a term-context matrix is enough to produce generalizations that rather spookily resemble human semantic models. Neural models do the same thing much better, but I'm still not prepared to see much in that beyond more evidence in favor of the distributional hypothesis. Which means that there's plenty of room to believe that what seems like reasoning on the part of the LLM is actually more akin to a projection from a sort of "holographic" representation of written artifacts of human reasoning.
That's not placing humans on a pedestal. That's just thinking hard about possible explanations for a phenomenon and then provisionally choosing the one I believe is most parsimonious. As any believer in scientific skepticism should strive to do.
>Well, like I was hinting at above, it's actually based on a close analysis of all the ways it doesn't quack like a duck
I doubt that, if your descriptions are anything to go by. "appropriately applies syllogism to arrive at correct conclusions" is just not something humans reliably do, so if that is the bar here then it's not worth anything, no matter what secret benchmark you think you've cooked up. Will you say humans cannot reason as well?
There's the classic example of the Wason Selection Task. Most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters.
>I would certainly be open to hearing a detailed explanation of how that kind of thing might actually support reasoning, but, absent that, pointing out that I don't understand how brains do it (which is true) is the pot calling the kettle black.
Not really. You're the one making a positive mechanistic claim here: that reasoning requires "actively selecting and manipulating a symbolic model at least semi independently of language generation" and that a decoder transformer therefore probably isn't doing it. The claim needs justification and it is once again another example of you assuming your claim and working backwards towards it.
"Next token prediciton" describes the training objective and the interface through which the model's computation is expressed. It does not tell you what computation the network must perform internally to minimize that objective. There is nothing contradictory about a system learning internal representations, manipulating them, and then using the result to predict the next token.
In fact, we already know transformers are capable of considerably richer internal computation than your description suggests. Much richer than we can even begin to understand.
We don't possess some mechanistic definition of human reasoning which we've opened the human skull and verified. We infer that humans reason largely from behaviour.
So if some other system exhibits some of the same behaviours, you need some non-question-begging criterion for why they cease to count as reasoning in that system. Otherwise the position just becomes unfalsifiable: when the model succeeeds on a novel reasoning problem, that's merely a "projection from a holographic representaion" (this is a meaningless statement by the way); when it fails, then you jump to claiming its evidence that wasn't/cannot reason at all (which is just silly and bad science).
>I actually want to turn the tables on your claim about humans being on a pedestal, and this is another spot where I believe I side with Hofstadter. We humans have a tendency to believe that certain behaviors are so special that the only way to accomplish a similar outwardly-visible behavior is by using similar underlying mechanisms. I see a healthy measure of chauvinism in that assumption.
Nobody is requiring the same mechanism. That's my point. I'm perfectly happy for transformers to reason in a drastically different way from humans (though i suspect it's not so drastic). You are the one requiring that it must have a particular internal form - one that cannot even be validated to exist - and then rejecting systems whose architecture doesn't obviously resemble that form.
>As I mentioned earlier, these LLMs are trained on vast collections of both the inputs and outputs of genuine human reasoning, and potentially have enough parameters to record a large proportion of that information in the form of probability distributions. And even something as crude as a singular value decomposition on a term-context matrix is enough to produce generalizations that rather spookily resemble human semantic models. Neural models do the same thing much better, but I'm still not prepared to see much in that beyond more evidence in favor of the distributional hypothesis. Which means that there's plenty of ...
A human wouldn't mindlessly reproduce the signature due to "not reasoning" and being intellectually lazy while doing the drawing. For the human, including the signature is more effort than omitting it; for the generative system, it appears the opposite is true. The point is to highlight that difference.
> “It’s like somebody attributed a quote to me that I didn’t say.”
> Katzenstein considers the reproduction of his signature by ChatGPT to be more than just a violation of intellectual property; to him, it’s closer to false impersonation. “[ChatGPT] is attaching my name to work that I do not endorse or like. It’s slop, and unlike the other slop that I’ve encountered, this is slop that’s pretending to be me.”
> “I’ve had people hack my credit card,” said Joe Dator, a New Yorker contributor for the past 20 years. “That feels like less of a violation than this. When they hacked my credit card, they didn’t dress up like me.”
So this has morphed from plagiarism and copyright infringement (bad) to impersonation (also bad, arguably worse, and maybe more provable in court). It’s chilling to think of the implications of having one’s signature attached to a document or to words that are not one’s own.
This is a pretty solid argument against people who argue that LLMs are more than just (very massive) next token predictors.
If there was any thought or underlying thought going on here not putting a signature (at least a real one) would be the right move, despite it being less likely. It would realize, while generating the pixels that eventually became a signature, that it shouldn't do that.
> This is a pretty solid argument against people who argue that LLMs are more than just (very massive) next token predictors.
This quote is a pretty solid argument that you need to understand the technology you’re trying to criticize better. This issue has nothing to do with LLMs. LLMs are not image generation models.
A LLM, at least, prompted and served this image. LLMs can ingest images. They actually can generate them as well but that probably wasn't done here.
In the course of the conversation a with chatgpt, this image was generated and served by an LLM. It clearly shouldn't have been by any sort of reasoning.
Nobody seems to care about AI automating coders out of a job.
If anyone can just prompt all their basic “information needs” however how sloppy, then what remains of the economy? Health care, child care, handyman?
Most people won’t even pay for ad free YouTube. I don’t think any software business can survive AI as a substitute good even if it’s inferior (and it might not be).
same here, been automating my work for 40 years starting with a data entry job I had in 1994 as an office temp, wrote a C program to do the whole thing for me, hung out on IRC all day where I made connections to get my first dot-com job
> If anyone can just prompt all their basic “information needs” however how sloppy, then what remains of the economy? Health care, child care, handyman?
That kinda describes a lot of “not the USA” developed countries since quite some time.
You may not like AI but their business model is a silly twitter-worthy dunk. This is obviously not their business model. I didn't know I could've plagiarized all this code myself the entire time!
This is one of the reasons I'm extremely unimpressed by complaints from openai and anthropic that other labs are "distilling" their models based on them... Basically: "You're training your model by running it again our own model which is itself a gargantuan copyright and content violation of a scale never seen before by humankind, HOW DARE YOU"
There are three things that have become very clear to me after the rise of generative AI:
1. The sheer amount of material on the internet that is "free to view but not free to use for any purpose" is the greatest resource of our time, and despite it being easy for individuals to take advantage of it (no one will take you to court for printing a newspaper comic and pinning it to your corkboard,) it's historically been difficult for corporations to exploit it (their best idea pre-AI is to encourage people to post it on social media walled-gardens where they can surround it with ads.)
2. The reason behind the impressive results of generative AI is because it exploits the above "free" resource, which is the greatest resource of our time. The reason behind the industry-wide push for AI and the insane amount of investment in it, is that they know it's their first real chance to exploit the greatest resource of our time. This is the gold rush.
3. Anthropomorphism is the wool that AI labs are pulling over legislators eyes, so they can pull off this heist. If you see training and inference as a black box, a process that consumes a copyrighted work (among others) and produces something very similar to the original work that also competes directly with it, is clearly something that's against the spirit of copyright. But if you (afraid of being judged a luddite) see AI as a little man inside the computer who is "learning" and "creating," how could you deny him? Especially if it would deny your jurisdiction access to the above gold rush. A lot of scientific-sounding AI communication is propaganda for this way of thinking, like the Anthropic J-space stuff, which stops just short of claiming AI is conscious, despite leading the reader to that conclusion.
You don’t need anthroporphism to draw the conclusion that using copyrighted material in model training is fair use. It extends pretty directly from existing fair use doctrine surrounding transformative uses of technology, in particular from Authors Guild v. Google, where the 2nd circuit ruled that creating a searchable database of copyrighted material (i.e. Google Books) was transformative, as long as the material presented to the end user was limited snippets of text and not the entirety. Bartz v. Anthropic explicitly cites that case as precedent
Google: scans all books, people can search for them, find little snippets, see the title and author, buy the book to read it
AI: scans all art, people can ask it to produce art they're searching for, no reference to the original or its author, people no longer pay artists/designers/...
I think there's a very obvious distinction there. The whole purpose of copyright is ensuring the financial viability of creating works. AI slop is a direct market substitute for the originals it ate up for training, so it goes directly against that. Meanwhile, search actually improves the reach of works and, well, snippets and summaries are somewhere in between...
In this instance the training data was New Yorker cartoons signed by Brendan Loper, and the material presented to the end user is New Yorker-style cartoons signed by Brendan Loper.
To me that seems a exceedingly broad definition of fair use.
Maybe the reproduction of the signature itself is a copyright violation, I’m not sure what the smallest unit of an existing work you can reproduce legally is, but reproducing a style is in no way a copyright offense. Artists do that all the time and there is existing federal case law specifically on the question of whether or not you can reproduce the style of New Yorker cover art (You can, see Steinberg v. Columbia Pictures)
Perhaps we need to create a new form of intellectual property to protect innovations in style. But that’s not copyright, which protects existing works from unauthorized reproduction. It seems like it would be very difficult to adjudicate though; how do you assess whether a style is truly unique to an author or artist?
This is exactly the sort of anthropomorphism I'm talking about.
Yes, human artists reproduce styles all the time, but implying this equivalence between what a human does when they recreate a style, and what an AI model has done when its output is suspiciously similar to a single artist's "contributions" to the training set is reductive since the mechanism is obviously completely different. It doesn't say anything about whether one ought to be allowed, just because the other is, but the assumption that it should is doing the legwork for those who benefit from AI as a copyright washing machine.
My point in saying that artists do this all the time is to say that there is a very long legal tradition which says that artists doing this does not violate copyright, rather than tediously listing the relevant cases.
My point is not that the process is the same. That’s not relevant to the law. What’s relevant is whether or not the material is a reproduction that substantially infringes upon the original.
I’m also not making an ought argument. If you want to change the law, write your congressman. What I’m arguing is that existing legal doctrine on copyright law very clearly does not protect mere ideas or styles.
As I understand the process is very relevant to the law: why things like clean-room reverse engineering can be necessary.
> What I’m arguing is that existing legal doctrine on copyright law very clearly does not protect mere ideas or styles.
Like it or not, it is anthropomorphizing to claim that an algorithm can have ideas or use styles, rather than just saying that the apparent "ideas" and "styles" in its output are a mathematical derivation of the human ideas and styles in the copyrighted material in its training set.
The narrative is constantly about "when AGI arrives." We're gonna have UBI, it's gonna solve human labor, it's gonna produce better and more music, cinema, art, and literature than all of human history before it. The AI bros from Altman all the way down to the YouTuber commenter say anything to keep the conversation off of the present. It's always the pie in the sky that is perennially almost here. The progress is happening "so fast." Next model update we're all gonna get luxury space communism.
AI is here. We've had a good long look at what it does and what it's used for, and it's not going to be something else. This is what it's for. It's for copyright washing other people's work (shitily). It's for astroturfing social media with product placement comments. It's for presidents to make videos of them dropping poop on protestors from airplanes. It's for souless "creators" to earn updoots from other bots for their "street photography" generated images of neon lights on puddles in Tokyo. It's rube goldberg automations that almost always accomplish nothing. It's fanatics claiming that it has multiplied their productivity by some incredible factor, but never showing the receipts, or when they do it's always something trivial like a calorie counter app.
This is it. This is the AI we've heard so much about. I'd say I can't wait for the hype to end, but after living through several cycles, I'm almost certain that whatever hype cycle emerges from the IT sector next will be even worse.
> If it is a hype, then one that produces lots of working code. Way faster than I can type it. And I am fast.
I covered that:
> It's fanatics claiming that it has multiplied their productivity by some incredible factor, but never showing the receipts, or when they do it's always something trivial like a calorie counter app.
There is no shortage of "working code" out there. GitHub is reportedly falling over from all the working code. Where's your business? Who's using it? Why aren't products better yet? It's a mirage. Your "working code" is the same thing as gen-AI "street photography" images. No one wants to see them, yet it's pumped out in vast enough amounts that it's choking the spaces that are ostensible for photography. The low quality, low investment nature of it excites the lazy wanna-bes who leave a trail of half-baked, soon-forgotten detritus behind them.
You’re making solid points but I do think that, at least, programming is changing to be around AI generation. Even if not much has changed in terms of visible output. AI may add some marginal GDP growth, nothing explosive. Switching from file cabinets to computers didn’t result in face melting productivity gains despite totally retooling how information flows through a business.
I don't disagree that it is useful. I, personally use it ping-pong coding problems before I break ground (although I never let opencode out of plan mode). It's marginally better than the plain old internet was for getting help before Google allowed SEO to ruin it, and I believe we're watching that edge get cannibalized in realtime as more and more LLM content ends up in the sources that LLMs synthesize in their answers.
If there are any face-melting characteristics to this whole thing it's how much money has been poured into it.
" It's marginally better than the plain old internet was for getting help"
Ok, so as an example, I have a old 2D game of mine. Custom written voxel engine for a shooter game with completely destroyable world. Works great, but 2D.
I always dreamed of making it 3D, but never found the time. It would have been a huuuge effort doing it on my own.
Some days ago I pointed fable at the engine and basically said, make it 3D.
And it did. Took some hours. Not by copying other voxel engines, but by making my engine 3D. (I know because I know there is no game on the market like this)
So yes, I knew the domain and thought long and hard about it and the core is handwritten and battle tested and that probably helped steering it in the right direction. But I did not wrote a single line of this new code. And it works awesome.
This is not what I call marginally improvement. But you do you.
IMO this doesn't really capture fully what AI is or does. It is possible to use AI to create artwork that has never been seen before and that is not derivative of an artist that already exists. I've used it myself for my own photos and drawings. It understands the basics of art like composition, lighting, the physics of things to some extent and all of this it uses in the art it creates.
The problem is that copyrighted material is intermingled with non-copyrighted material in a way where it's not obvious how to solve it. But I think in this case, AI is so powerful that we should (gasp) cut it some slack. This would be the perfect example of throwing the baby out with the bathwater if OpenAI were to be sued out of oblivion.
It seems like all the criticisms of Gen AI and LLM seems to concentrate on OpenAI and their products over products from anthropic and others.
I am pretty sure that this faking of signature can be done by Gemini, claude as easily as chatgpt.
Why would ChatGPT be sued exactly? They didn’t publish the picture, they did what the user asked. The user is responsible because they directed the creation and publishing. The user is the entity who should be sued. Or dmca’d. Or whatever.
Think if the user commissioned the art from an outsourced creative shop nobody has heard of. Then they published it. They wouldn’t go after the creative shop, they would go after the publisher.
(I am just addressing publishing here, training on the artist’s works is a different, well discussed issue)
It is certainly not trademark infringement, I can say that for sure -- no reasonable person would prompt ChatGPT to create a "new yorker style cartoon" and then think the result is produced by the actual cartoonist in mere seconds for their viewing pleasure. So there's no confusion in the marketplace.
The user could cause confusion in the marketplace of course, but that would be her doing, not the app's. Surely we can all agree that suing adobe illustrator for facilitating trademark infringement of logomarks and such would be silly?
It could be copyright infringement, which should drive home how absurd copyright is as a concept. Everyone's all up in arms about Anthropic reporting a user to the police today -- imagine if the thing they were reporting was that she had written a sacred symbol in her personal notebook...
> no reasonable person would prompt ChatGPT to create [...] and then think the result is produced by the actual cartoonist
There's also a narrower situation to consider, where the user describes something they "want"--implicitly to find--but the system generates a fraudulent one instead.
In that case ChatGPT would be committing a trademark violation, at least within a nation of laws rather than lobbying.
The article is evidence there is confusion in the marketplace - people are contacting the cartoonists about the fraudulent works.
Now the argument can be where does the infringement land - does it land with the person who generates the cartoon, fails to remove the signature and uploads it to instagram? Maybe. But I don't think we should be giving these AI companies yet another get out of jail "oops you made a machine that does a bad thing" card.
I also firmly believe there are contexts in which merely producing the trademark and attaching it to something is enough to be infringement but I'm not going to do the legal legwork to get there to your satisfaction, sorry.
As I said, I am talking about the publishing, not the training. If it produces his signature, which basically claims he created the art, then that is the line he believes was crossed. I am saying that it is up to the user to understand when those lines are crossed. The user could have done this in photoshop as well without ai, would we blame photoshop? Training is a separate argument, it is not what the article tackles.
It’s their hardware and their web response. Especially more heinous if it’s being served as part of a subscription. I don’t think it’s functionally the same as me opening Microsoft Paint and recreating pixel-for-pixel a New Yorker artist’s signature.
I don't see that as a reasonable argument unless you're claiming the user was lying:
> In the comments section of her post, she wrote she had simply asked ChatGPT to make “a New Yorker-style cartoon.”
If you commissioned me to record music onto a CD for you, and then I put in the credits that Jimi Hendrix recorded the guitar parts without you asking, it seems pretty reasonable that I should get in trouble for that rather than you.
I don’t think she was lying. She created a cartoon and published it irresponsibly. In my opinion, that makes it her fault. And I think that AI is not going to be found culpable for issues like this in the future. The user needs to understand they can’t just publish whatever it creates. There are open source AI models that would have near zero culpability and in that case I believe very, very clearly it will be the users responsibility. And I think that will extend to paid models as well.
OpenAI being bad actor would not have been a problem if OpenAI suffered relevant consequences, like everyone else would have in that scenario. Or at least this is how I understood this.
At most you might try to argue negligence, but there's no sign that OpenAI is being a malicious bad actor here.
Adding unwanted elements to image generation is undesirable by all parties, including OpenAI.
It's not a good analogy cause no one sues each other for little hammer injuries. It's more like the guy who gets wacked saying 'hey what the heck man that's my distinct hammer design but you made a machine that makes almost precisely my exact hammer and I hold a patent for this'...(or something like that)
"You've made a machine that can be tuned to produce almost precisely my exact hammer and people who use the machine get freaked out that I can sue them."
It's a preemptive middle finger to the fuckers who downvote shit they can't handle, like with the other comments that were already gray when I posted mine
I've seen photorealistic generations add (distorted but still somewhat recognisable) watermarks too, because that's what the training data had.
Everything is a derivative work, and always has been. AI is just making that salient fact so much more visible, and now everyone who believes in the delusion of Imaginary Property is scared at that truth revealing itself.
Incidentally, this is also what young humans learning to draw will do. They start by copying what they've seen.
Sometimes they do, but more often what they do is copy someone else’s style and perspective, then they call it their own without any attribution whatsoever. They also tend to draw a lot of copyrighted material, and they even sell those unlicensed drawings for profit.
>I'm certain if I drew a cartoon and used the signature that looks just like a real cartoonists', I would get sued and liable for the damage.
Would you? Always? Suppose I hate Obama drone striking people, so I made a satirical cartoon of him signing an executive order to "bomb brown people" or whatever, affixing his signature[1] to that image. Would that get me in trouble, even if the image was clearly satirical? What if someone takes that, then passes it as non-satire?
There's nothing illegal unless someone seriously thinks Obama signed it. The easiest example would be his signature on his wikipedia page. Clearly he didn't sign that page, nor (probably) did he authorize it.
That's the whole point of hypotheticals? If we're just discussing the common case, the conclusion is so obvious that no serious analysis happens and everything becomes vibes based. "Is torrenting fine?" would have been met with something along the lines of "yes, copyright is just an artificial monopoly created by the government and harms innovation", "information should be free", etc. Now people have done a 180 because it's evil corporations doing it so the vibes have totally reversed. They may have a point about the vibes of evil corporations training AI models then selling back the models for $$$, but vibes is no way to run a legal system.
>Even if you posted on Twitter or something, nobody is coming after you.
Are you sure about that one? Are you saying if you posted say, a racist cartoon on twitter, and you forge my signature onto the thing, I can't come after you?
I don't know about the US but here in Germany the category for this is personality rights. The entire point of a signature is to establish authenticity, using someone's likeness or identity without consent will get you into deep trouble even without money being exchanged.
Not true in the US. It is true that most plaintiffs prioritize commercial copies, but you can obtain statutory damages even if there are no actual damages from the copying.
Many companies and artists simply allow technically infringing fan art etc. to exist without enforcement.
I remember seeing some documentary about an art forger who donated his works to museums where the authorities were profoundly frustrated that they couldn't do anything precisely because he didn't get any gain.
No one asked, but I'll analyze the differences between these two situations. Landis didn't impersonate any living people. It wasn't done against a backdrop of living people losing their jobs. What he made was still art, in that it was both a display of technical skill and worth discussion. Slop is neither. From now on, his forgeries will be clearly labeled as forgeries any time they're displayed. That's easy to do with a physical painting, but impossible when an image has been shared across platforms.
This reminds me my early attempts to use GitHub Copilot when it just straight added some guy's name in a javadoc copyright note in the code it generated.
314 comments
[ 4.1 ms ] story [ 50.1 ms ] threadWhistleblower mysteriously found dead after exposing industrial scale Copyright abuse. Isomorphic plagiarism of $9Tn in FOSS and user content is the product. =3
> The New York Times article cites Stanford University law professor Mark Lemley, who disagreed that generative AI services violate copyright law, and intellectual property attorney Bradley Hulbert, who said a new law might be necessary to settle the question of legality.
> Months after Balaji's death, which attracted significant public attention, Hulbert told Fortune magazine that Balaji's essay "[reads like] the argument of a really smart non-lawyer who read up on the subject but does not have a thorough understanding".
If there's some kind of industrial-scale intimidation campaign that's stopping IP lawyers from litigating the case of their lifetime, that's an even bigger story than OpenAI taking out a hit on somebody. It seems like they're agreeing that the copyright abuse was never hidden, and it's sufficiently transformative enough that nobody could argue it's illegal.
One could argue nobody should be allowed to claim it. It just exists.
Even if the person can't claim the copyright of the image produced they ARE responsible for the use of their tools and what they do with the output.
In this case, they released an image with someone else's signature on it. That is wrong, the person should take the blame for that.
The person releasing the image may take it up with the AI service that their tooling led them into making such a mistake. But good luck with that in court...
Somebody “made something.” But just because you do something doesn’t mean you get to claim whole ownership of it and get to sign it with your name. Plenty of examples in life.
LLM generated picture with artists signature. That's no problem.
Person takes this picture and proceeds to share it around is the problem.
What annoys me is when "LLM hacks website!". Uh, it just went and did that completely unprompted? Nope.
Somebody ran it on a computer somewhere, gave the LLM some kind of prompt+access combination - however precise or vague - and THEN the LLM did a thing.
Power tools are well known for making human bits go missing. But nobody says "The angle grinder did it on it's own! Nobody is at fault!"
Maybe the person wasn't trained. Maybe it's the wrong tool for the job. Maybe the manufacturer of the tool is at fault for selling a product without appropriate safety guards. But the LLMs aren't doing self sustaining thinking for themselves just yet.
It’s funny I talk about this a lot. I haven’t quite worked out my metaphor, but it feels kind of akin to somebody writing a poem for the first time or drafting a short story. They drink a bunch of coffee, hammer it out through the middle of the night, then blast it out to the world expecting praise and adoration (whether they’ve written before or not). But then they get upset when people say it’s not complete or it looks rushed. Indignant even.
Despite how quickly LLMs turn out things and how little effort you can put in to get something out of them, people seem to have this weird cognitive dissonance about it. They feel like you are attacking them and then passionately defend their tools. But they didn’t really put that much effort in, so why are they bothered when you treat it that way?
The 'bug' here is whatever post-processing step or system prompt is in place to steer the model away from doing this.
I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.
its somewhat funny that math people are in a conundrum as to support or not support but this might partially be because some wish to believe that math itself is and can be useful and therefore accelerating is good
but the art people have no such delusions so they’re just strictly against
imo proof writing is more akin to art than coding/tech but…
It is very tiring to say “I don’t necessarily disagree with you about AI ‘art’, but in my field—which you do not understand, and in which the underlying build process is often not the creative output—AI presents very real productivity gains” for the umpteenth time.
I am skeptical of there being sufficient data to build “ethical” training datasets, and I’m confident that much of the same contingent will (somewhat rightfully) argue that ‘second-generation’ copyrighted AI material has already irreversibly made its way into every modern dataset.
But sure, there's um, an ethical way of doing that?
1. Only use open source/CC compliant assets.
2. Acquire rights/licenses to any datasets that do not fit #1. e.g. the Google deal with Reddit for 60m/yr.
3. Offer programs to have creatives willingly submit their data, with some sort of residual output based on the number of times their assets are sampled.
4. If all that is still not enough, hire creatives to create assets for you. This is something Spotify did recently with "ghost artists"[0]. The intentions here are suspect, but a non-consumer facing artist providing work for an LLM wouldn't have the same ethical dilemmas
5. Lastly, if all that still isn't enough: governmental programs to either provide grants, subsidies, or more outreach to get the ball rolling.
Would this cost tens, hundreds of billions of dollars? Yes. But clearly, that was not a barrier to entry for the industry anyway. So we can chalk this down to the personality of leadership or the wider culture of modern big tech
[0]: https://harpers.org/archive/2025/01/the-ghosts-in-the-machin...
Like, legally, I'm sure Reddit had the right to sell it, but probably over half their content was written before ChatGPT was ever announced. The TOS allowing reddit to make "derivative works" was largely understood to mean things like cropping photos, using your viral post in an ad, or maybe auto-translating your comment.
Being able to prove such gains in better products would be a start. And an emphasis on how it assists existing engineers/mathmaticians/researchers, not that any accomplishment made with AI assistance is "AI solves problem".
I don't know whatever happened to "words are cheap". I guess it literally made money to say words, so that adage is false for the time being.
>I am skeptical of there being sufficient data to build “ethical” training datasets
Well if all those scam job ads paying 100/hr to create AI training content was not a scam and instead the approach from the start, there may have been a chance to bridge that gap ethically. The industry chose to break things and is trying to act mad that people are mad at all the broken stuff.
These results are entirely a consequences of the actions chosen. And I don't believe there was ever an honest consideration of there being ethical training datasets. They just thought they could brute force society with fearmongering and bribes. The BOTD was already low in the beginning but completely gone now.
There are actual models trained on ethical datasets but they are obviously not very high powered. If companies with the resources of an anthropic or openai were doing it (ha) it would be more feasible
That’s not a justification. If a company were poisoning the water to your home as a byproduct, would you be satisfied if they told you “we don’t necessarily disagree with you about polluting the water, but in our field—which you do not understand, and in which the underlying build process is often not the water pollution—what we’re doing presents very real productivity gains”?
> I am skeptical of there being sufficient data to build “ethical” training datasets
Then you don’t build any. What fucked up world we live in where people think it’s OK to be unethical because they want something and can’t think of any other way to do it. What monumentally selfish rotten babies.
https://en.wikipedia.org/wiki/Whataboutism
Other things being bad doesn’t make it OK that this thing is and no one is saying that.
My argument was purposefully general, you’re the one who chose to bring it back to AI. Stop arguing in bad faith.
AI having negative externalities is not the sole deciding factor in deciding to eradicate it
then my argument: negative externalities are tolerated in many areas where we deem the topic is of enough value that the externalities are managed. there are many examples of things with negative externalities, much much larger than those of AI, that aren't being considered for eradication.
so your argument is implicitly, "AI is not worth these negative externalities"
which is a common opinion among people who dont use AI at all.
> If a company were poisoning the water to your home as a byproduct, would you be satisfied if they told you “we don’t necessarily disagree with you about polluting the water, but in our field—which you do not understand, and in which the underlying build process is often not the water pollution—what we’re doing presents very real productivity gains”?
no but I also wouldn't declare that whatever it is that company does should have its entire industry eradicated. Poor industrial practices can be mitigated while not abandoning the product being manufactured.
By your logic, they don't particularly contribute to scientific productivity/advances: it's not like you can use stable diffusion to generate an architectural diagram.
GenIA for images and videos is only really used to copy artstyles, make fake menus and create misinformation so wouldn't you agree that eradicating this part would not only preserve your described "useful applications" but also allow providers to allocate more resources to "useful" and "scientific" AI?
I know that local models have become decent enough that this isn't really doable, but I'm only proposing this thought experiment because you seem to think people are saying this should be all or nothing. But the truth is, if you get a genie to wipe away just diffusion models, the entire internet gets much better (or at least back to regular levels of bad), no one loses anything, and a whole lot of people stop complaining/campaigning against the productivity gains in your field.
- prove that you had the rights for all of your training data
- open source the model
Give the labs a 3 month grace period in which to comply, so competition can persist even with dubiously sourced data, but the people can't be locked away from derivatives of their contributions for any significant amount of time.
I disagree that they can be separated. Practically, I think they can't. Because the mere invention of new tools inspires even more AI advancement and that in turn will cause the other side (artistic side) to degenerate even more.
I'm anti-LLM all the way, 100%, no exceptions. Zero tolerance.
While coders may care about the craft (and I do), it's not as if the value of my code is in the exact variable names I chose.
Meanwhile there’s endless discussions about the right way to name variables (not too shot, not too long, try to be self documenting, but not to the point of putting types in the name like we used to), and people definitely get judged by their variable names, it’s one of the first things someone will point out when they look at a codebase. “Why are all the variables a single letter this is bad code!”
The current models intelligence depends on massive training dataset of essentially stolen data
google OTOH already had a lot of this dataset in their possession (e.g. Google Books etc), still questionably licensed for how they used it, but not quite as bad. They did apparently break through NYT paywalls and stuff like that though, still theft.
Code can be art, and copyright/plagiarism is real. It sort of boils down to how much it bothers us.
For certain values of "my own".
https://lemmy.ca/post/63438408
Given that they can reliably do this, I'd think it should be trivial for the harness to automatically insert a "review the image for anything that looks like an artist's signature or other blatant indicator of plagiarism, and fix it" pass.
There's a program for that on your computer, it's called 'true'.
This is completely silly. If you don’t think LLMs can reason, you’ve either never used them to do tasks that require reasoning, or you don’t understand enough to recognize what’s involved in the responses you get.
In this case it’s clearly the latter, because you’re confusing image generation models with LLMs. There are very big differences between the two. No-one is claiming that image generation models are capable of reasoning.
I didn't think stage magicians could manufacture animals ... until I saw a guy pull a rabbit from his hat.
The thing about a generative language model that’s trained from a massive but unknown corpus is, it’s practically (if not theoretically) impossible to evaluate the extent to which data leakage contributes to any particular output.
But I would argue that, as things currently stand, “sophisticated engine for approximately querying a pastiche of the results of human reasoning that comprise its training corpus” remains a more parsimonious explanation than “it’s doing actual reasoning” for how this neural network architecture produces the phenomena we’ve been observing.
Well if the thing can find and fix bugs in something that is using non-mainstream stuff that is surely not in it's training dataset, that's better than a rubber duck already. Whether it has soul is a different question of course.
As popular as I know the rhetorical tactic is on both sides of these discussions about LLMs, I’d still thank you not to strawman me.
Perhaps you could argue that “appropriately applies syllogism to arrive at correct conclusions” is too high a bar to set, but I don’t think it would be fair to call it a “goofy-ass”, “non-standard” or “fluid” element of a reasoning capacity assessment.
I don't think the bar for an actual reasoner can possibly be 'always appropriately applies syllogism to arrive at correct conclusions', because in that case nobody in the world is an actual reasoner. And if your bar were 'sometimes appropriately applies syllogism to arrive at correct conclusions', it's hard to understand why the current generation of AIs doesn't meet it; they are clearly capable of doing so, at least to all outward appearances. (Maybe you think their apparently successful demonstrations of reasoning are illusions, but again, you would need to define what counts as "actual reasoning" vs. a superficially convincing simulation of it.)
More generally, it’s important to keep the burden of proof in the right place. The idea that this particular neural network architecture is performing reasoning tasks is really quite extraordinary, and we should demand an unusually compelling case to be made before accepting it.
Compare to the example of Douglas Hofstadter and computer chess. In the late 70s he predicted that computers would need general intelligence to compete at the top levels in chess. When computers did start competing at that level, he did not automatically assume chess programs are intelligent. He took a look at how they are implemented, decided that the idea that that particular algorithm was doing abstract reasoning was implausible, and instead re-evaluated his assessment of what it takes to compete with a top tier human at chess.
I tend to align with Hofstadter’s approach to the question. Like I pointed at in a cousin comment, arguments that what we see represents reasoning tend to hinge on a common logical fallacy.
You have misunderstood the 'burden of proof' quite badly here. The extraordinary claim here is that what looks like a duck, quacks like a duck, acts like a duck is in fact not a duck even though you are simultaneously completely incapable of proving it is not a duck.
Just because you have placed humans on a pedestal does not mean you are not the one making the extraordinary claim. Hofstadter is entitled to his opinion and while you are free to follow his 'approach', you'd do well to consider that neither you nor Hofstadter in fact have any idea whatsoever how 'abstract reasoning' is implemented in even the systems you are certain posses it. Don't be confused that you are doing anything more than begging the question here.
I would certainly be open to hearing a detailed explanation of how that kind of thing might actually support reasoning, but, absent that, pointing out that I don't understand how brains do it (which is true) is the pot calling the kettle black.
I actually want to turn the tables on your claim about humans being on a pedestal, and this is another spot where I believe I side with Hofstadter. We humans have a tendency to believe that certain behaviors are so special that the only way to accomplish a similar outwardly-visible behavior is by using similar underlying mechanisms. I see a healthy measure of chauvinism in that assumption.
As I mentioned earlier, these LLMs are trained on vast collections of both the inputs and outputs of genuine human reasoning, and potentially have enough parameters to record a large proportion of that information in the form of probability distributions. And even something as crude as a singular value decomposition on a term-context matrix is enough to produce generalizations that rather spookily resemble human semantic models. Neural models do the same thing much better, but I'm still not prepared to see much in that beyond more evidence in favor of the distributional hypothesis. Which means that there's plenty of room to believe that what seems like reasoning on the part of the LLM is actually more akin to a projection from a sort of "holographic" representation of written artifacts of human reasoning.
That's not placing humans on a pedestal. That's just thinking hard about possible explanations for a phenomenon and then provisionally choosing the one I believe is most parsimonious. As any believer in scientific skepticism should strive to do.
I doubt that, if your descriptions are anything to go by. "appropriately applies syllogism to arrive at correct conclusions" is just not something humans reliably do, so if that is the bar here then it's not worth anything, no matter what secret benchmark you think you've cooked up. Will you say humans cannot reason as well?
There's the classic example of the Wason Selection Task. Most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters.
>I would certainly be open to hearing a detailed explanation of how that kind of thing might actually support reasoning, but, absent that, pointing out that I don't understand how brains do it (which is true) is the pot calling the kettle black.
Not really. You're the one making a positive mechanistic claim here: that reasoning requires "actively selecting and manipulating a symbolic model at least semi independently of language generation" and that a decoder transformer therefore probably isn't doing it. The claim needs justification and it is once again another example of you assuming your claim and working backwards towards it.
"Next token prediciton" describes the training objective and the interface through which the model's computation is expressed. It does not tell you what computation the network must perform internally to minimize that objective. There is nothing contradictory about a system learning internal representations, manipulating them, and then using the result to predict the next token.
In fact, we already know transformers are capable of considerably richer internal computation than your description suggests. Much richer than we can even begin to understand.
We don't possess some mechanistic definition of human reasoning which we've opened the human skull and verified. We infer that humans reason largely from behaviour.
So if some other system exhibits some of the same behaviours, you need some non-question-begging criterion for why they cease to count as reasoning in that system. Otherwise the position just becomes unfalsifiable: when the model succeeeds on a novel reasoning problem, that's merely a "projection from a holographic representaion" (this is a meaningless statement by the way); when it fails, then you jump to claiming its evidence that wasn't/cannot reason at all (which is just silly and bad science).
>I actually want to turn the tables on your claim about humans being on a pedestal, and this is another spot where I believe I side with Hofstadter. We humans have a tendency to believe that certain behaviors are so special that the only way to accomplish a similar outwardly-visible behavior is by using similar underlying mechanisms. I see a healthy measure of chauvinism in that assumption.
Nobody is requiring the same mechanism. That's my point. I'm perfectly happy for transformers to reason in a drastically different way from humans (though i suspect it's not so drastic). You are the one requiring that it must have a particular internal form - one that cannot even be validated to exist - and then rejecting systems whose architecture doesn't obviously resemble that form.
>As I mentioned earlier, these LLMs are trained on vast collections of both the inputs and outputs of genuine human reasoning, and potentially have enough parameters to record a large proportion of that information in the form of probability distributions. And even something as crude as a singular value decomposition on a term-context matrix is enough to produce generalizations that rather spookily resemble human semantic models. Neural models do the same thing much better, but I'm still not prepared to see much in that beyond more evidence in favor of the distributional hypothesis. Which means that there's plenty of ...
- LLM's can reason
- everything an LLM does is the result of reasoning
This demolishes only the latter point, which as far as I know has no supporters.
I think these talking points are there to hype up the technology or otherwise excuse the mis-allignment (i.e. consumer hostility).
We are probably some years away from a strong LLM with natively visual output.
> Katzenstein considers the reproduction of his signature by ChatGPT to be more than just a violation of intellectual property; to him, it’s closer to false impersonation. “[ChatGPT] is attaching my name to work that I do not endorse or like. It’s slop, and unlike the other slop that I’ve encountered, this is slop that’s pretending to be me.”
> “I’ve had people hack my credit card,” said Joe Dator, a New Yorker contributor for the past 20 years. “That feels like less of a violation than this. When they hacked my credit card, they didn’t dress up like me.”
So this has morphed from plagiarism and copyright infringement (bad) to impersonation (also bad, arguably worse, and maybe more provable in court). It’s chilling to think of the implications of having one’s signature attached to a document or to words that are not one’s own.
If there was any thought or underlying thought going on here not putting a signature (at least a real one) would be the right move, despite it being less likely. It would realize, while generating the pixels that eventually became a signature, that it shouldn't do that.
This quote is a pretty solid argument that you need to understand the technology you’re trying to criticize better. This issue has nothing to do with LLMs. LLMs are not image generation models.
In the course of the conversation a with chatgpt, this image was generated and served by an LLM. It clearly shouldn't have been by any sort of reasoning.
We can all try really hard to pretend that's not the business model, but that's totally the business model.
If anyone can just prompt all their basic “information needs” however how sloppy, then what remains of the economy? Health care, child care, handyman?
Most people won’t even pay for ad free YouTube. I don’t think any software business can survive AI as a substitute good even if it’s inferior (and it might not be).
somehow I still keep having work to do!
That kinda describes a lot of “not the USA” developed countries since quite some time.
1. The sheer amount of material on the internet that is "free to view but not free to use for any purpose" is the greatest resource of our time, and despite it being easy for individuals to take advantage of it (no one will take you to court for printing a newspaper comic and pinning it to your corkboard,) it's historically been difficult for corporations to exploit it (their best idea pre-AI is to encourage people to post it on social media walled-gardens where they can surround it with ads.)
2. The reason behind the impressive results of generative AI is because it exploits the above "free" resource, which is the greatest resource of our time. The reason behind the industry-wide push for AI and the insane amount of investment in it, is that they know it's their first real chance to exploit the greatest resource of our time. This is the gold rush.
3. Anthropomorphism is the wool that AI labs are pulling over legislators eyes, so they can pull off this heist. If you see training and inference as a black box, a process that consumes a copyrighted work (among others) and produces something very similar to the original work that also competes directly with it, is clearly something that's against the spirit of copyright. But if you (afraid of being judged a luddite) see AI as a little man inside the computer who is "learning" and "creating," how could you deny him? Especially if it would deny your jurisdiction access to the above gold rush. A lot of scientific-sounding AI communication is propaganda for this way of thinking, like the Anthropic J-space stuff, which stops just short of claiming AI is conscious, despite leading the reader to that conclusion.
AI: scans all art, people can ask it to produce art they're searching for, no reference to the original or its author, people no longer pay artists/designers/...
I think there's a very obvious distinction there. The whole purpose of copyright is ensuring the financial viability of creating works. AI slop is a direct market substitute for the originals it ate up for training, so it goes directly against that. Meanwhile, search actually improves the reach of works and, well, snippets and summaries are somewhere in between...
In the US it is "To promote the Progress of Science and useful Arts".
Of course, it was also supposed to be a limited-time monopoly rather than perpetual...
To me that seems a exceedingly broad definition of fair use.
Perhaps we need to create a new form of intellectual property to protect innovations in style. But that’s not copyright, which protects existing works from unauthorized reproduction. It seems like it would be very difficult to adjudicate though; how do you assess whether a style is truly unique to an author or artist?
This is exactly the sort of anthropomorphism I'm talking about.
Yes, human artists reproduce styles all the time, but implying this equivalence between what a human does when they recreate a style, and what an AI model has done when its output is suspiciously similar to a single artist's "contributions" to the training set is reductive since the mechanism is obviously completely different. It doesn't say anything about whether one ought to be allowed, just because the other is, but the assumption that it should is doing the legwork for those who benefit from AI as a copyright washing machine.
My point is not that the process is the same. That’s not relevant to the law. What’s relevant is whether or not the material is a reproduction that substantially infringes upon the original.
I’m also not making an ought argument. If you want to change the law, write your congressman. What I’m arguing is that existing legal doctrine on copyright law very clearly does not protect mere ideas or styles.
> What I’m arguing is that existing legal doctrine on copyright law very clearly does not protect mere ideas or styles.
Like it or not, it is anthropomorphizing to claim that an algorithm can have ideas or use styles, rather than just saying that the apparent "ideas" and "styles" in its output are a mathematical derivation of the human ideas and styles in the copyrighted material in its training set.
AI is here. We've had a good long look at what it does and what it's used for, and it's not going to be something else. This is what it's for. It's for copyright washing other people's work (shitily). It's for astroturfing social media with product placement comments. It's for presidents to make videos of them dropping poop on protestors from airplanes. It's for souless "creators" to earn updoots from other bots for their "street photography" generated images of neon lights on puddles in Tokyo. It's rube goldberg automations that almost always accomplish nothing. It's fanatics claiming that it has multiplied their productivity by some incredible factor, but never showing the receipts, or when they do it's always something trivial like a calorie counter app.
This is it. This is the AI we've heard so much about. I'd say I can't wait for the hype to end, but after living through several cycles, I'm almost certain that whatever hype cycle emerges from the IT sector next will be even worse.
I am curious, your type seems to become a rare species - I suppose you have never worked with a modern coding agent recently?
Otherwise you have noticed also here many, if not the majority expressing they rarely if ever write code anymore?
If it is a hype, then one that produces lots of working code. Way faster than I can type it. And I am fast.
I covered that:
> It's fanatics claiming that it has multiplied their productivity by some incredible factor, but never showing the receipts, or when they do it's always something trivial like a calorie counter app.
There is no shortage of "working code" out there. GitHub is reportedly falling over from all the working code. Where's your business? Who's using it? Why aren't products better yet? It's a mirage. Your "working code" is the same thing as gen-AI "street photography" images. No one wants to see them, yet it's pumped out in vast enough amounts that it's choking the spaces that are ostensible for photography. The low quality, low investment nature of it excites the lazy wanna-bes who leave a trail of half-baked, soon-forgotten detritus behind them.
Show us the receipts.
And I mean no wonder github is drowning in garbage, now that anyone can "code"
"The low quality, low investment nature of it excites the lazy wanna-bes who leave a trail of half-baked, soon-forgotten detritus behind them."
But would you also call Linus Torvalds a lazy wannabe?
If there are any face-melting characteristics to this whole thing it's how much money has been poured into it.
Ok, so as an example, I have a old 2D game of mine. Custom written voxel engine for a shooter game with completely destroyable world. Works great, but 2D.
I always dreamed of making it 3D, but never found the time. It would have been a huuuge effort doing it on my own.
Some days ago I pointed fable at the engine and basically said, make it 3D.
And it did. Took some hours. Not by copying other voxel engines, but by making my engine 3D. (I know because I know there is no game on the market like this)
So yes, I knew the domain and thought long and hard about it and the core is handwritten and battle tested and that probably helped steering it in the right direction. But I did not wrote a single line of this new code. And it works awesome.
This is not what I call marginally improvement. But you do you.
It's also forgery as a service.
The problem is that copyrighted material is intermingled with non-copyrighted material in a way where it's not obvious how to solve it. But I think in this case, AI is so powerful that we should (gasp) cut it some slack. This would be the perfect example of throwing the baby out with the bathwater if OpenAI were to be sued out of oblivion.
Layering and placement of copyrighted materials to evade copyright.
Think if the user commissioned the art from an outsourced creative shop nobody has heard of. Then they published it. They wouldn’t go after the creative shop, they would go after the publisher.
(I am just addressing publishing here, training on the artist’s works is a different, well discussed issue)
OpenAI give lip service to the idea of not producing others' intellectual property - go ask it to explicitly make a picture of the genie from Aladdin.
The user could cause confusion in the marketplace of course, but that would be her doing, not the app's. Surely we can all agree that suing adobe illustrator for facilitating trademark infringement of logomarks and such would be silly?
It could be copyright infringement, which should drive home how absurd copyright is as a concept. Everyone's all up in arms about Anthropic reporting a user to the police today -- imagine if the thing they were reporting was that she had written a sacred symbol in her personal notebook...
There's also a narrower situation to consider, where the user describes something they "want"--implicitly to find--but the system generates a fraudulent one instead.
In that case ChatGPT would be committing a trademark violation, at least within a nation of laws rather than lobbying.
Now the argument can be where does the infringement land - does it land with the person who generates the cartoon, fails to remove the signature and uploads it to instagram? Maybe. But I don't think we should be giving these AI companies yet another get out of jail "oops you made a machine that does a bad thing" card.
I also firmly believe there are contexts in which merely producing the trademark and attaching it to something is enough to be infringement but I'm not going to do the legal legwork to get there to your satisfaction, sorry.
> In the comments section of her post, she wrote she had simply asked ChatGPT to make “a New Yorker-style cartoon.”
If you commissioned me to record music onto a CD for you, and then I put in the credits that Jimi Hendrix recorded the guitar parts without you asking, it seems pretty reasonable that I should get in trouble for that rather than you.
You think the user asked for a signature?
Negligence at scale, perhaps.
should Adobe be sued?
(sorry not sorry for going against the bandwagon, let the downvotes as disagreement commence!)
AI models are generating the image
Pretty simple.
They could have fixed this if they cared.
Even your contrarianism is unoriginal.
Which means a slap on the wrist at the moment I suppose.
[1]: https://www.reuters.com/business/media-telecom/disney-univer...
Everything is a derivative work, and always has been. AI is just making that salient fact so much more visible, and now everyone who believes in the delusion of Imaginary Property is scared at that truth revealing itself.
Incidentally, this is also what young humans learning to draw will do. They start by copying what they've seen.
The same should apply to LLM vendors.
Would you? Always? Suppose I hate Obama drone striking people, so I made a satirical cartoon of him signing an executive order to "bomb brown people" or whatever, affixing his signature[1] to that image. Would that get me in trouble, even if the image was clearly satirical? What if someone takes that, then passes it as non-satire?
[1] https://en.wikipedia.org/wiki/File:Barack_Obama_signature.sv...
There's nothing illegal unless someone seriously thinks Obama signed it. The easiest example would be his signature on his wikipedia page. Clearly he didn't sign that page, nor (probably) did he authorize it.
If you just draw a cartoon with a fake signature, nobody is coming after you. Even if you posted on Twitter or something, nobody is coming after you.
You'd have to be fraudulently selling it in a book of cartoons or on coffee mugs or something.
Are you sure about that one? Are you saying if you posted say, a racist cartoon on twitter, and you forge my signature onto the thing, I can't come after you?
I don't know about the US but here in Germany the category for this is personality rights. The entire point of a signature is to establish authenticity, using someone's likeness or identity without consent will get you into deep trouble even without money being exchanged.
The product is the output, not the tokens.
Many companies and artists simply allow technically infringing fan art etc. to exist without enforcement.
No one asked, but I'll analyze the differences between these two situations. Landis didn't impersonate any living people. It wasn't done against a backdrop of living people losing their jobs. What he made was still art, in that it was both a display of technical skill and worth discussion. Slop is neither. From now on, his forgeries will be clearly labeled as forgeries any time they're displayed. That's easy to do with a physical painting, but impossible when an image has been shared across platforms.