To be fair, the title is the same but this article is really about a study of the openness characteristics of different AI models, which is new. This link had the comparison of different models: https://opening-up-chatgpt.github.io/
Also, there's an error in that table, the first model "Bloomz" does not have an open source license, on the HF page for the model it shows a RAIL license which is a restrictive license that limits you to "approved" uses. The license link goes to the github repo where the code is released under Apache 2.0. I didn't review the rest but I'd say it calls the ranking into question.
Agreed it's annoying, but on a certain level also not entirely wrong. In a software 2.0 [0] world the weights are functionally the code in that it is what gets you from input to output.
Open weights or something similar would be better though
Weights almost entirely encapsulate the behavior of the model. Whatever structural analogy you want to use, they are closest in spirit to the core functional algorithm of traditional software.
Of note, if you're interested in helping participate in the discussion of what an open source AI would actually look like, the Open Source Initiative is looking for your help:
In there somewhere in there where they are asking for input? It presents itself that way but it seems the only way to "help" is to propose a presentation under their call for speakers: https://sessionize.com/deepdiveai
As I read the process, at this point they're primarily looking for "conversation starters" (my phrase, not theirs) in the form of presentations, as well as attendees to discuss at a number of in-person events this year (I'll be at the one at All Things Open in Raleigh NC in October). I think once they've got something a little more substantial to review, they'll have a more robust "public comment"-type process.
I'm sure you could also offer a meta-comment on the process itself.
It's also a necessary goal in order for these models to be more broadly adopted.
We've seen too many examples of bias in the training data set manifesting in ways that actively discriminate against people. Which is unethical and in many places illegal.
And having copyrighted material and sexual content in your model will simply open you up to lawsuits as is happening right now between authors and OpenAI. Not sure that is a position most startups want to be in.
College students don't want to be criminally charged with theft for pirating an $800 college textbook either. I'm not giving business advice, I'm giving humanity and knowledge propagation advice.
As I mentioned, reducing bias is good so long as it doesn't introduce more bias elsewhere. The ultimate goal of course being a 100% bias free model.
Nah, these are foundation models. Companies want to be able to put in guard rails that are applicable to their application, not start with a model lobotomized according to US tech company values. The censorship is about telling people how to think, like it always is, not for the good of the people using the models.
Yup. Advance the foundation models as far as possible to create a representation of the internet/human experience and place safeguards on top.
Don't want your LLM to be used to create erotic fanfiction? Instead of gutting the LLM, just put a detector on the query and answers feeding into the LLM. Thankfully, with a non-neutered model you have access to a tool than can be used to perform such detection...
We really need to somehow separate a bias towards accuracy as distinct from some bias towards say, a sports team.
Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones.
This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished people can do consistently well in.
Listening to everybody and incorporating their thoughts is only going to lead to wrong answers.
Being selective is the key here unless you genuinely want say, answers about to space to involve aliens and UFOs - because way more people believe in that then there are qualified PhD astrophysicists in the world.
Similarly, way more people believe in vaccine conspiracy theories then there are people with significant viral epidemiology backgrounds.
Why is it a binary choice? "Most students would answer X, due to this common misconception about Y".
As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with.
Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.
An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real.
If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever.
The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity.
Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking.
We would have been done with that by now if this were the case.
Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis
Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.
If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason.
Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.
> Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.
Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet.
The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."
Exactly. Especially when there's some hustle behind lying to people. Phony medical advice is the classic one here. People do it to sell their sugar pills or do some Peter Popoff style healing touch hustle.
Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated.
The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material.
To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things.
If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit.
A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game.
For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.
Just because it is a probability doesn't mean that it's strictly the probability derived from frequency in your training corpus. If you just used that as your model, it would not be nearly as good. Things like reasoning, a world model, and other heuristics can be used to generate said probabilities internally.
All that being said, who is to say that humans aren't just looking at probabilities when they think?
When you dig into it, there are very few truths. Who "won" the last US election? Where did covid come from?
Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.
That's not true though. Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go.
To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students.
The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion.
When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Youtube comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's say, 10 youtube comments saying one thing and 5 papers in the Lancet saying another. The youtube comments don't become correct by volume here.
Are you saying that the way people naturally speak follows no rules whatsoever?
Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing.
No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own.
To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.
I think the claim isn't that "language has no rules" but rather that the "rules" are far more context dependent and subject to change over time than would ordinarily be implied by the use of the word "rules".
Even at the lowest level of phonemes to morphemes, accents introduce a huge amount of variety, and it doesn't really get more orderly as you increase the level of abstraction.
An grammar so no because no rules yes Have. then its they're simple she Great understand your of problem in yes?
Or did I break some rules there?
We're not talking Oxford commas here. There are fundamental structures.
For instance, If I made that "are there fundamental structures" then it's a question.
Basic grammar is very real and not some subjective ephemeral thoughtstuff with wildly different opinions. It's so obvious that you don't even question it.
That's the way most things are - how to operate a faucet, close a window, put on a shirt, open a cabinet, use a fork... Most things in life aren't controversial.
Even the controversial ones like what's art and what's music, it's just a small periphery that's questioned. A symphony played by an orchestra is music, a painting in a museum is art - even these subjective categories substantially have extremely wide agreement.
True controversy is an outlier. (Ie, is 4'33" by John Cage music?)
You can claim whatever you want but with just things that's just exercising the freedom to be wrong.
The point stands. Things are often depicted too binary and simplistic. Many subjective things also get smuggled in as if they were objective to hide their ideology and the word "truth" is used to stop people from questioning things.
I'm totally onboard with that.
At the same time, when talking about the physical world, most things are not like that. Humans for instance, get around by moving limbs called feet and not flapping feathered wings like a bird. This is the level of obvious I'm talking about.
The secondary point was that many things are this level of obvious and also specialized. For example: how does the animal cacomantis castaneiventris get around? Unless you have some kind of specialized knowledge you wouldn't know that's a type of bird.
The internet is awash in unspecialized people opining on specialized knowledge that is only controversial to the untrained eye in the same way that students don't get As on every exam they take. You need to actively work around people talking out of their bounds if you want your training data to be accurate
> Copyrighted material, sexual content, political opinions, throw it all in and release it please!
Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative works that replace them from it?
I can't imagine what goes through the head of someone who thinks they don't have the right to say no. They don't have the right to be listened to or the right to use state power to enforce a colloquial notion of IP and rights with no basis in law or fact, but they surely have the right to say "No!"
In all seriousness, they're afraid because they aren't very pleasant. AI will write things people will like and enjoy hearing. Preachy artists these days just make works they think people ought to want, and ought to like. Getting moralism out of art will be a great technological accomplishment.
Moralism! In art! Heaven forfend. How dare that art have a message, right? Never in history, not once, before the Woke Tens and, I guess, the Woker Twenties did somebody have an opinion and encode it in allegory. It is a purely novel phenomenon and all known media before this dark age was absolutely without message and without any editorial lens to determine what is and is not to be depicted--for that, of course, would communicate values and ideas, and art simply did not do such a thing.
More seriously: if you have this attitude for-reals and aren't Poe's Lawing me, perhaps you need a little more preaching aimed your way because your conception of art and the civic society is the closest thing I can think of to a humanities position being objectively wrong.
The ideological soma of agree-with-you AI being desirable is scary enough stuff. Removing communication from art being a "great technological accomplishment" is fucking dystopic.
I want to see the world where this happens, where everything that made life enjoyable is corporatized, standardized, optimized, one-dimensionalized. Since it will happen over the course of your lifetime, you'll get to see it happen - as things that were fulfilling cease to be, an ennui you can't escape sets in because you built every wall of your cage one by one, and all you can do is regret. And then the Black Mirror helmet comes off your head, everyone else points and laughs at you, and then puts the helmet back on.
I wonder if there's a future title like "AI Model Artist", "AI Model Contributor", "AI Model Author", "AI Model Creator" or something for creators who contribute to the models in use.
While I love we can all immediately benefit from previous works, I feel for the countless artists and creators who's work has been integrated into models with zero compensation. There are already outstanding lawsuits seeking reparations, but this is new legal territory and there will be a lot to learn and decide in the future.
I think if AI art damages the commons to an extreme extent, there's a real possibility that AI companies pay artists to make whatever the fuck they want to keep the training data coming.
And some would argue that billion dollar companies like OpenAI, Meta etc should figure out a way to compensate the people who created the content they depend on.
Making the uncensored, unbiased, un-"aligned" results available to all for a fair price is compensation enough, I think. That's what we should be pushing for.
Whether demon or genie, any notion of putting this tech back in the bottle is a non-starter. As is turning it into a giant money grab for the copyright industry.
They're not. They have a copyright. Practicing artists usually benefit professionally from maintaining a public portfolio. Data being public is also notably not a license to use it for whatever purpose you want. Have some respect.
I have a copyleft, for all the good it does my work. Copyright protects artists from nothing, and leaving it uncontested harms the consumer more than the artists.
> Data being public is also notably not a license to use it for whatever purpose you want.
Under the Fair Use doctrine, it very well could be. It was when Google indexed every book they could buy, to the disdain of the Author's Guild and the titleholders they represented.
> Have some respect.
I will not respect an authority that forces me to rent digital content.
Pretty much everything nowadays is copyrighted, by omitting such materials, what are you really left with?
LLM is a tool much like the internet is a tool. Yes, someone can use it to steal, but stealing is against the law.
Instead of encoding a criminal justice system into an LLM by omitting the possibility of stealing an artists work or omitting the knowledge of physics so someone can't learn how to build a bomb, we should instead just prosecute people for using it in that way intentionally.
How often do people get prosecuted for ripping off an artists style? The criminal justice system "hated artists" long before LLMs and it's not the responsibility of the tech. companies to rectify that in my opinion.
You're not addressing the massive abuse of the commons this still represents. If artists don't have the right to tell you to fuck off for using their work in training data, they're less likely to publicly show that work, which hurts them because they become less visible and hurts the AI because the training data gets worse.
They do have an option; they can choose to not publish their work. By digitizing your creation, you are creating a version of your work that can be distributed at practically-free prices with little effort. If that undermines or undervalues the art you make enough, you can choose not to share it.
Even Open Source advocates don't really have the right to stop companies from using Open code. The license discourages it, but everyone from Tesla to Nintendo has been caught violating it's terms. Publishing stuff on the open web has always had consequences, unfortunately.
Artists are generally pretty encouraging to people entering the field and using their stuff as reference for new artists. That actually contributes to art. It's definitely not the same thing as a massive tech corporation trying to automate their livelihoods, but please keep making this flawed argument analogizing two completely different processes.
> It's definitely not the same thing as a massive tech corporation trying to automate their livelihoods, but please keep making this flawed argument analogizing two completely different processes.
Ah yes, the famous massive tech corporation behind stable diffusion.
> Now tell me how artists were not stealing from each other.
Key words being "from each other". AI only takes, it doesn't give anything back, it doesn't inspire. Their ultimate goal is to absorb the entire human history of art and then displace millions of people who received nothing from this transaction they were forced into, just so that some billionaire can afford another yacht.
I have no problems with AI models, but if you want to use art, writing, code, etc. for training, you should restrict your use to public domain works, ask for consent, or commission it. Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.
> AI only takes, it doesn't give anything back, it doesn't inspire.
Totally wrong. There is even a well known counter example: dream-like videos generated by ai wasn't something we had before.
This statement tells me you never used it.
> just so that some billionaire can afford another yacht
You are heavily mistaken, that the scenario where ai learning isn't considered as fair use of material. In this case, only megacorps will be able to trains their own AI by spending billions in content.
> Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.
You are constructing your own story while ignoring what is happening.
Adobe and Dall-e models were trained with datasets they mostly had rights over. OpenAI partnered with Shutterstock, and Adobe have their own photo stock.
Big tech companies didn't had problem looking for image content.
Stable Diffusion on the other hand is open source & open research, but didn't had any rights on most of the image they trained.
We know who copyright is meant to protect, and it's not artists: it's Disney's shareholders. Pay writers and actors a fair cut, and then maybe people will take seriously that copyright is for artists.
>Could we stop celebrating how tech is going to steal everyone's copyrighted works
Copyright has a fair-use exception for transformative works. It is difficult to look at the LLM's of the day and think "No, they have not taken the copywritten works and transformed them into something completely new."
There is no hate of artists. I don't know how you get from "It's not copyright violation" to "I hate artists". This is a matter of existing law. It is of course unsettled, the fair-use interpretation not yet tested in courts. But the same goes for affirming it's a violation of copyright.
These questions will have their day in court, and until then there is no need to make arguments in an inflammatory tone that that borders on personal attack. In the meantime, maybe engage in a conversation about how copyright law would need to be changed to account for this new technology, not condemn people who see a reasonable interpretation of existing law the differs from your own.
So I can have a giant database of all the most recent copyrighted works, on my home computer, as long as I claim I'm using it for training a model? And if I happen to listen to some music or watch some movies too, who's going to know?
I think an argument could be made that it's fine for an LLM to learn from copyrighted works, but maybe it should have to go to the library to do it. Having your own human-accessible copy of those works (at home, or at Meta), doesn't sound as acceptable to me.
Sure, I think you’re right, it’s reasonable to expect a company to at least buy a single copy of each book (or whatever it eats) it ingests if the work could not otherwise be legally obtained for free. Or something reasonable along those lines
It’s not Microsoft’s fault that people are generally required to buy a book in order to read it. You could say the same thing about any business with a high capital cost required to get up and running. Want to start a supermarket? That’s many $Millions to plan, buy/rent land, build the facility, stock the shelves, etc. Starting any business costs money, and some cost a lot of money. Suppliers of the materials or feedstock required to start them aren’t to blame. If I want to fine tune a LLM to be an expert on the Dune book series, should I be allowed to download a copy without paying, or should I have to pay $8 for a copy before I feed it to the model? I’m sure the publisher would want me to pay some ridiculous amount but that is a different argument than requiring the purchase of a single copy that anyone would have to do to get access to the content.
I believe LLMs should be allowed to read/view/consume content and learn from it even if that content has a copyright.
We phrase it like somehow the material is being copied into the LLM, but that’s not what it’s doing. It’s building a neural graph from the experience of consuming that content.
What would the world be like if humans couldn’t learn, train the weights of the interconnects of their neural tissue, from any material with a copyright?
My thinking “the enemy gate is down” when considering the tokens “Ender’s Game” is my recalling a learned association of those tokens to the given token string.
My knowing that doesn’t strip the copyright. My telling someone the meaning and context of the phrase generally doesn’t strip the copyright away from Orson Scott Card. I’m not reproducing his work but my knowledge of it. And it’s dependent on what I do with that knowledge and how if I’ve violated his copyright.
We are prosecuting the LLMs for possessing fragments of knowledge. And we’re assuming that the recall of some of those fragments means a copy of that work is in fact contained within the weights.
A mathematical transformation of the data is not enough to qualify as a transformative work. Saving a copy-written work in a lossy compression does not negate the copywrite.
There is a lot of mental gymnastic, to submit an artwork publicly on the internet, allowing everyone to copy your arstyle, make derivative art of it, have other learn from it, but if ever a machine 'learn' from it, it's "stealing". It's not because you don't like a derivative work, that the derivative work is "stealing" your content.
Saying it's stealing is wrong, it's lying to get your point accross.
You blame "how tech is going to steal everyone copyrighted works", yet, they were already in the tech world: the internet.
It's a lot of mental gymnastic to think machine learning = human learning. Especially on HN, where people should understand scale matters a lot in real world.
It's generally consider ok to sell your fanart on comic festivals. But do you think it's okay that Disney starts selling fanart of One Piece without the publisher's permission?
A car and a pair of legs both move you from A point to B point. So why do laws treat automobiles and pedestrians so differently?
> Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is a no-no.
What other option would there be though? It seems like a binary field. You can either leave it wholly unaligned, or you can attempt to align the model, and you will introduce your own bias as a side effect.
In the future, and in fact in the present, there is already a "gray market" for unaligned models. There will surely be market for these and they will sell just any other item on this market.
> Yes, reducing bias in the models is a noble goal
“Bias” is not a unidimensional thing (or even one with a meaningful magnitude); you can’t reduce bias, you can make it more transparent (better documented) so that you can evaluate appropriateness for particular uses.
While I am not a big fan of the crew that pushes “alignment” as the main issue with regard to AI for other reasons, that’s a particularly good term for what is important – aligning the biases of the AI with the usage intent. “Reducing” bias makes sense only with regard to a specific usage intent.
devils advocate: if it wasn't subscribers only, google's crawler would be stealing it right now to train an ai with, ie, illegally creating derivative works based on their articles
Maybe there's a market in the future for "AI-restricted" content. You can send your payment/tokens/etc. to access the content and the user is bound by a no-AI-clause. Because this content would be rare and can't be found via models, it offers a niche market.
I wanted to delete the other link, but hackernews doesn't allow me to do so.
Also, I need to figure out why the link is subscriber-only. It was not intentional.
"Mark Dingemanse, a coauthor of this report, had a particularly strong assessment of the Llama 2 model: "Meta using the term `open source' for this is positively misleading: There is no source to be seen, the training data is entirely undocumented, and beyond the glossy charts the technical documentation is really rather poor. We do not know why Meta is so intent on getting everyone into this model, but the history of this company's choices does not inspire confidence. Users beware.""
Bad company scary, I guess. Normally I'm sympathetic to those types, but given how Llama works without a network connection I'm not sure what I have to fear. The "history of this company's choices" has generally been pretty good with AI, so it sounds like speculation on their part.
> He is an associate professor in Language and Communication at the Centre for Language Studies of Radboud University Nijmegen. Dingemanse obtained a MA degree in African Languages and Cultures at Leiden University in 2006, and a PhD degree in arts in 2011 at Radboud University Nijmegen
This co-author? Given the relevance of their academic background, I'd take their opinion on "the dangers of using LLMs released by the untrustworthy Meta" with a grain of salt.
Since their academic background is so relevant (language studies and cultures being the exact domain of the impact of LLMs on society), I don't see why a grain of salt is needed.
If their academic background were relevant, they would have provided specific negative outcomes that may result from the usage of Llama 2, rather than a vague "the history of this company's choices does not inspire confidence".
What, of the many open source software releases by that company in the ML/AI field "does not inspire confidence"?
> What, of the many open source software releases by that company in the ML/AI field "does not inspire confidence"?
Didn't Facebook get it's start by Zucc populating it with his classmates without their knowledge or consent?
> Zuckerberg found himself brought before the Administrative Board for breaching security, violating copyrights and violating individuals’ privacy by using students’ online facebook photos without permission.
For some background on the comment, see the following paper.
Title: Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators
Authors: Andreas Liesenfeld, Alianda Lopez, Mark Dingemanse
Abstract: Large language models that exhibit instruction-following behaviour represent one of the biggest recent upheavals in conversational interfaces, a trend in large part fuelled by the release of OpenAI's ChatGPT, a proprietary large language model for text generation fine-tuned through reinforcement learning from human feedback (LLM+RLHF). We review the risks of relying on proprietary software and survey the first crop of open-source projects of comparable architecture and functionality. The main contribution of this paper is to show that openness is differentiated, and to offer scientific documentation of degrees of openness in this fast-moving field. We evaluate projects in terms of openness of code, training data, model weights, RLHF data, licensing, scientific documentation, and access methods. We find that while there is a fast-growing list of projects billing themselves as 'open source', many inherit undocumented data of dubious legality, few share the all-important instruction-tuning (a key site where human annotation labour is involved), and careful scientific documentation is exceedingly rare. Degrees of openness are relevant to fairness and accountability at all points, from data collection and curation to model architecture, and from training and fine-tuning to release and deployment.
It seems Dr Dingemanse understands the problems introduced by so-called "tech" companies, e.g., Google.
"For basic visitor statistics I use Matomo, an excellent open source alternative for Google Analytics. IP addresses are anonymized and data never leaves the server."
I think we have forgotten the foundation of open source which was free software which was about empower the user.
Instead of all these complicated criteria, there are only 2 criteria that get to the heart of the question:
1. Can you run this on your own hardware
2. Can you use this directly or tweak this as a community to make porn without a bunch of barriers
This gets to the heart of the matter: Can you run the software without having to be dependent on the company, and can you modify the software to do something that the company may not want you to do with it.
If it is able to pass both tests, it is functionally "open source"
By that criteria, ChatGPT is definitely not open but Llama is closer
It's shocking to me how many people in tech feel completely entitled to intellectual property that took someone years to master a skill to make. But talk about releasing a proprietary codebase and suddenly they want the lawyers involved because that actually threatens their livelihood.
Is that actually a commonly held position? I've seen IP abolishonosts here, and I've seen people argue the merits of proprietary software, but I don't get the impression that those are generally the same people.
They’re straw-manning, programmers are the best sharers in the world. Open source software has lead the drive for open source learning and information in general.
Right here the comment says they prefers Meta to keep the secret sauce as long as it allows they to (inderectly) access the copyrighted material etc. And the replies below are generally positive. So it's not a straw-man, at least in this thread, at all.
It should be noted, that they're explicitly taking this position because they think it will result in more capabilities. Not because of a respect of intellectual property.
So we should probably find a way for artists to be able to publicly share their work without some ass holes in San Francisco creating a massively profitable system based on that work that disrupts their livelihoods.
This is exactly right. Almost everything you see on the internet is copyrighted, even if you're allowed to view it for free. Even FOSS and CC-licensed content is copyrighted. It's just as copyrighted a proprietary codebase. But for some reason, you don't see coding LLMs trained on proprietary code, eg. you don't see Microsoft training Copilot on private GitHub repos. It all smells very hypocritical, like these companies feel entitled to harvest free labour from the internet to make their proprietary AI models, but they're not willing to do the same with their own.
You are aware that, between the unix wars, the spats with microsoft, and the general people who grew up ripping music/games off of random places, you're probably in one of the most hostile places to IP... pretty much anywhere, right?
The entire subculture that this site takes its name from is hostile to IP.
I mean, except for the start-up founders that hang around here, but this site is mostly kept up as a way for them to recruit it seems like (and maybe a way to build good will).
Edit: To be clear, I'm not necessarily saying you're wrong, but you could probably pick a more sympathetic argument from a strategic perspective. What you've done is kinda the equivalent to going to the Vatican and arguing that something is a threat to idol worship. You might be right, but a good chunk of your audience wants that.
129 comments
[ 4.0 ms ] story [ 190 ms ] threadhttps://news.ycombinator.com/item?id=36820122
https://news.ycombinator.com/item?id=36806536
https://news.ycombinator.com/item?id=36791671
https://news.ycombinator.com/item?id=36783019
Open weights or something similar would be better though
[0] https://karpathy.medium.com/software-2-0-a64152b37c35
https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...
var a = b * 3 + 2;
You're telling me the * 3 + 2 part isn't code and only var a = b is code?
https://opensource.org/deepdive/
Is there another way to contribute?
I'm sure you could also offer a meta-comment on the process itself.
I don't want a sanitized LLM, and I don't have $60M lying around to train my own.
Copyrighted material, sexual content, political opinions, throw it all in and release it please!
Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is a no-no.
Maybe I just got added to a list somewhere for having this opinion.
It's also a necessary goal in order for these models to be more broadly adopted.
We've seen too many examples of bias in the training data set manifesting in ways that actively discriminate against people. Which is unethical and in many places illegal.
And having copyrighted material and sexual content in your model will simply open you up to lawsuits as is happening right now between authors and OpenAI. Not sure that is a position most startups want to be in.
As I mentioned, reducing bias is good so long as it doesn't introduce more bias elsewhere. The ultimate goal of course being a 100% bias free model.
Don't want your LLM to be used to create erotic fanfiction? Instead of gutting the LLM, just put a detector on the query and answers feeding into the LLM. Thankfully, with a non-neutered model you have access to a tool than can be used to perform such detection...
Which do broadly align with the values across most of the world.
But if you want to build something that has different values then go ahead.
But you can't expect companies to be complicit in doing something which is unethical or illegal.
Satire?
Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones.
This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished people can do consistently well in.
Listening to everybody and incorporating their thoughts is only going to lead to wrong answers.
Being selective is the key here unless you genuinely want say, answers about to space to involve aliens and UFOs - because way more people believe in that then there are qualified PhD astrophysicists in the world.
Similarly, way more people believe in vaccine conspiracy theories then there are people with significant viral epidemiology backgrounds.
This pattern is true in every field.
As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with.
Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.
An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real.
If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever.
The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity.
Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking.
We would have been done with that by now if this were the case.
Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis
If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason.
Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.
Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet.
The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."
Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated.
The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material.
To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things.
If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit.
A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game.
For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.
All that being said, who is to say that humans aren't just looking at probabilities when they think?
Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.
To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students.
The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion.
When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Youtube comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's say, 10 youtube comments saying one thing and 5 papers in the Lancet saying another. The youtube comments don't become correct by volume here.
It's not some giant equal playing field.
As a linguistic descriptivist, hahahahahahahaha.
Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing.
No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own.
To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.
Even at the lowest level of phonemes to morphemes, accents introduce a huge amount of variety, and it doesn't really get more orderly as you increase the level of abstraction.
Or did I break some rules there?
We're not talking Oxford commas here. There are fundamental structures.
For instance, If I made that "are there fundamental structures" then it's a question.
Basic grammar is very real and not some subjective ephemeral thoughtstuff with wildly different opinions. It's so obvious that you don't even question it.
That's the way most things are - how to operate a faucet, close a window, put on a shirt, open a cabinet, use a fork... Most things in life aren't controversial.
Even the controversial ones like what's art and what's music, it's just a small periphery that's questioned. A symphony played by an orchestra is music, a painting in a museum is art - even these subjective categories substantially have extremely wide agreement.
True controversy is an outlier. (Ie, is 4'33" by John Cage music?)
You can claim whatever you want but with just things that's just exercising the freedom to be wrong.
I'm totally onboard with that.
At the same time, when talking about the physical world, most things are not like that. Humans for instance, get around by moving limbs called feet and not flapping feathered wings like a bird. This is the level of obvious I'm talking about.
The secondary point was that many things are this level of obvious and also specialized. For example: how does the animal cacomantis castaneiventris get around? Unless you have some kind of specialized knowledge you wouldn't know that's a type of bird.
The internet is awash in unspecialized people opining on specialized knowledge that is only controversial to the untrained eye in the same way that students don't get As on every exam they take. You need to actively work around people talking out of their bounds if you want your training data to be accurate
Could you expand on that a bit?
Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative works that replace them from it?
In all seriousness, they're afraid because they aren't very pleasant. AI will write things people will like and enjoy hearing. Preachy artists these days just make works they think people ought to want, and ought to like. Getting moralism out of art will be a great technological accomplishment.
More seriously: if you have this attitude for-reals and aren't Poe's Lawing me, perhaps you need a little more preaching aimed your way because your conception of art and the civic society is the closest thing I can think of to a humanities position being objectively wrong.
The ideological soma of agree-with-you AI being desirable is scary enough stuff. Removing communication from art being a "great technological accomplishment" is fucking dystopic.
While I love we can all immediately benefit from previous works, I feel for the countless artists and creators who's work has been integrated into models with zero compensation. There are already outstanding lawsuits seeking reparations, but this is new legal territory and there will be a lot to learn and decide in the future.
A society without copyright would be a much poorer one, and humans figured that out a long time ago.
That was before "Attention Is All You Need" and the subsequent work on LLMs that grew out of it. Things are different now.
Copyright may need to be updated to cater for the new world of AI but that doesn't mean as a concept it is evil.
https://www.theguardian.com/books/2023/jul/05/authors-file-a...
Whether demon or genie, any notion of putting this tech back in the bottle is a non-starter. As is turning it into a giant money grab for the copyright industry.
> Data being public is also notably not a license to use it for whatever purpose you want.
Under the Fair Use doctrine, it very well could be. It was when Google indexed every book they could buy, to the disdain of the Author's Guild and the titleholders they represented.
> Have some respect.
I will not respect an authority that forces me to rent digital content.
They are very obviously asking you to respect the individual artists.
LLM is a tool much like the internet is a tool. Yes, someone can use it to steal, but stealing is against the law.
Instead of encoding a criminal justice system into an LLM by omitting the possibility of stealing an artists work or omitting the knowledge of physics so someone can't learn how to build a bomb, we should instead just prosecute people for using it in that way intentionally.
How often do people get prosecuted for ripping off an artists style? The criminal justice system "hated artists" long before LLMs and it's not the responsibility of the tech. companies to rectify that in my opinion.
Even Open Source advocates don't really have the right to stop companies from using Open code. The license discourages it, but everyone from Tesla to Nintendo has been caught violating it's terms. Publishing stuff on the open web has always had consequences, unfortunately.
Ah yes, the famous massive tech corporation behind stable diffusion.
Key words being "from each other". AI only takes, it doesn't give anything back, it doesn't inspire. Their ultimate goal is to absorb the entire human history of art and then displace millions of people who received nothing from this transaction they were forced into, just so that some billionaire can afford another yacht.
I have no problems with AI models, but if you want to use art, writing, code, etc. for training, you should restrict your use to public domain works, ask for consent, or commission it. Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.
Totally wrong. There is even a well known counter example: dream-like videos generated by ai wasn't something we had before. This statement tells me you never used it.
> just so that some billionaire can afford another yacht
You are heavily mistaken, that the scenario where ai learning isn't considered as fair use of material. In this case, only megacorps will be able to trains their own AI by spending billions in content.
> Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.
You are constructing your own story while ignoring what is happening.
Adobe and Dall-e models were trained with datasets they mostly had rights over. OpenAI partnered with Shutterstock, and Adobe have their own photo stock. Big tech companies didn't had problem looking for image content. Stable Diffusion on the other hand is open source & open research, but didn't had any rights on most of the image they trained.
Copyright like many regulatory devices can be used for good, evil and everything in between.
Copyright has a fair-use exception for transformative works. It is difficult to look at the LLM's of the day and think "No, they have not taken the copywritten works and transformed them into something completely new."
There is no hate of artists. I don't know how you get from "It's not copyright violation" to "I hate artists". This is a matter of existing law. It is of course unsettled, the fair-use interpretation not yet tested in courts. But the same goes for affirming it's a violation of copyright.
These questions will have their day in court, and until then there is no need to make arguments in an inflammatory tone that that borders on personal attack. In the meantime, maybe engage in a conversation about how copyright law would need to be changed to account for this new technology, not condemn people who see a reasonable interpretation of existing law the differs from your own.
I think an argument could be made that it's fine for an LLM to learn from copyrighted works, but maybe it should have to go to the library to do it. Having your own human-accessible copy of those works (at home, or at Meta), doesn't sound as acceptable to me.
We phrase it like somehow the material is being copied into the LLM, but that’s not what it’s doing. It’s building a neural graph from the experience of consuming that content.
What would the world be like if humans couldn’t learn, train the weights of the interconnects of their neural tissue, from any material with a copyright?
At the very least I think LLMs trained on data that the trainer does not own or have rights to use in that manner should not be copyrightable.
My thinking “the enemy gate is down” when considering the tokens “Ender’s Game” is my recalling a learned association of those tokens to the given token string.
My knowing that doesn’t strip the copyright. My telling someone the meaning and context of the phrase generally doesn’t strip the copyright away from Orson Scott Card. I’m not reproducing his work but my knowledge of it. And it’s dependent on what I do with that knowledge and how if I’ve violated his copyright.
We are prosecuting the LLMs for possessing fragments of knowledge. And we’re assuming that the recall of some of those fragments means a copy of that work is in fact contained within the weights.
You blame "how tech is going to steal everyone copyrighted works", yet, they were already in the tech world: the internet.
It's generally consider ok to sell your fanart on comic festivals. But do you think it's okay that Disney starts selling fanart of One Piece without the publisher's permission?
A car and a pair of legs both move you from A point to B point. So why do laws treat automobiles and pedestrians so differently?
That's why I have put it in quotes.
> But do you think it's okay that Disney starts selling fanart of One Piece without the publisher's permission?
It's why it's called "fair use". Fanart isn't the only case of fair use. Parody for example allows it.
And AFAIK, no court yet have decided that AI "learning" is not fair use or what the limite are.
Why did you put one of those in quotes and not the other?
What other option would there be though? It seems like a binary field. You can either leave it wholly unaligned, or you can attempt to align the model, and you will introduce your own bias as a side effect.
In the future, and in fact in the present, there is already a "gray market" for unaligned models. There will surely be market for these and they will sell just any other item on this market.
“Bias” is not a unidimensional thing (or even one with a meaningful magnitude); you can’t reduce bias, you can make it more transparent (better documented) so that you can evaluate appropriateness for particular uses.
While I am not a big fan of the crew that pushes “alignment” as the main issue with regard to AI for other reasons, that’s a particularly good term for what is important – aligning the biases of the AI with the usage intent. “Reducing” bias makes sense only with regard to a specific usage intent.
https://archive.is/GmVKl
Also the article seems to be written by an LLM anywway, so it wouldn't matter in the end.
https://en.wikipedia.org/wiki/Mark_Dingemanse
This co-author? Given the relevance of their academic background, I'd take their opinion on "the dangers of using LLMs released by the untrustworthy Meta" with a grain of salt.
If their academic background were relevant, they would have provided specific negative outcomes that may result from the usage of Llama 2, rather than a vague "the history of this company's choices does not inspire confidence".
What, of the many open source software releases by that company in the ML/AI field "does not inspire confidence"?
Didn't Facebook get it's start by Zucc populating it with his classmates without their knowledge or consent?
> Zuckerberg found himself brought before the Administrative Board for breaching security, violating copyrights and violating individuals’ privacy by using students’ online facebook photos without permission.
https://www.thecrimson.com/article/2004/6/10/mark-e-zuckerbe...
Title: Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators
Authors: Andreas Liesenfeld, Alianda Lopez, Mark Dingemanse
Abstract: Large language models that exhibit instruction-following behaviour represent one of the biggest recent upheavals in conversational interfaces, a trend in large part fuelled by the release of OpenAI's ChatGPT, a proprietary large language model for text generation fine-tuned through reinforcement learning from human feedback (LLM+RLHF). We review the risks of relying on proprietary software and survey the first crop of open-source projects of comparable architecture and functionality. The main contribution of this paper is to show that openness is differentiated, and to offer scientific documentation of degrees of openness in this fast-moving field. We evaluate projects in terms of openness of code, training data, model weights, RLHF data, licensing, scientific documentation, and access methods. We find that while there is a fast-growing list of projects billing themselves as 'open source', many inherit undocumented data of dubious legality, few share the all-important instruction-tuning (a key site where human annotation labour is involved), and careful scientific documentation is exceedingly rare. Degrees of openness are relevant to fairness and accountability at all points, from data collection and curation to model architecture, and from training and fine-tuning to release and deployment.
https://arxiv.org/abs/2307.05532
https://arxiv.org/pdf/2307.05532.pdf
"For basic visitor statistics I use Matomo, an excellent open source alternative for Google Analytics. IP addresses are anonymized and data never leaves the server."
https://markdingemanse.net/credits
Instead of all these complicated criteria, there are only 2 criteria that get to the heart of the question:
1. Can you run this on your own hardware
2. Can you use this directly or tweak this as a community to make porn without a bunch of barriers
This gets to the heart of the matter: Can you run the software without having to be dependent on the company, and can you modify the software to do something that the company may not want you to do with it.
If it is able to pass both tests, it is functionally "open source"
By that criteria, ChatGPT is definitely not open but Llama is closer
Right here the comment says they prefers Meta to keep the secret sauce as long as it allows they to (inderectly) access the copyrighted material etc. And the replies below are generally positive. So it's not a straw-man, at least in this thread, at all.
The entire subculture that this site takes its name from is hostile to IP.
I mean, except for the start-up founders that hang around here, but this site is mostly kept up as a way for them to recruit it seems like (and maybe a way to build good will).
Edit: To be clear, I'm not necessarily saying you're wrong, but you could probably pick a more sympathetic argument from a strategic perspective. What you've done is kinda the equivalent to going to the Vatican and arguing that something is a threat to idol worship. You might be right, but a good chunk of your audience wants that.