I agree that open source or just “open” is the wrong term here although it’s hard to apply it even to a project where the training data and everything else are shared. “Free” or “libre” might be closer since the end product is free to use and modify. It’s still fuzzy though, are weights code or are they an API? Oddly enough they seem to be the embodiment of the interchangeability of code and data perhaps due to neural networks being closer to mathematics than software normally is.
it's not fuzzy. weights are binary blobs. we know what binaries are, source code gets compiled into binaries. photoshop is a binary. firmware are binary blobs. that llama3.ggml outputs text when using a wrapper program vs photoshop launches a program directly doesn't make anything fuzzy.
Not really, those weights are the same ones that are there during training getting modified by backprop. You can take them and continue training the model in order to produce different behaviour. You can also modify them directly like with LoRA or steering. You can also make bigger modifications like expanding the layers or removing nodes. It’s not a frozen binary, those numbers aren’t obfuscated or optimized into some special purpose form that prevents or makes modification difficult.
And I can edit binaries with binedit. Reverse engineering is a lost art, but reversing binaries with Ida pro and patching binaries is a time honored skill. It's a shame it's usually used to patch out license checks but there have been some truely impressive reverse engineering on video game ROMs for which the source is not available so we're left with modifying the binary. or as we're looking at it, the model.
But you’re not reversing anything with weights. They aren’t altered at all. There’s no secret human readable code, just training/test data and hyperparameters. Even if they did give you access to the exact data set they used there’s not much chance you could do anything with it without millions of dollars worth of compute time and even then they’ve already trained the model on that data so the best you could do is start from scratch and see if you could get a better result in which case your time is better spent on building your own data set and RLHF process. Like what value do you think you’re going to get anyways?
I am not sure why people keep pretending like this is possible or keep asking it to happen.
You know that distributing all the data would be illegal copyright infringement, right? Also, its not like they could distribute the (petabytes?) worth of data to every rando who asks.
Do you just not want open weight models to exist, and you are using a fake call for it to be more open, so as to get this stuff all shut down, or what?
Because anyone who actually supports open "anything" would understand this, instead of pedantically arguing about the definition of open source and by suggesting things that both aren't possible and would make the release of these models much less likely.
Perhaps a good solution would be to form a trust responsible for storing and distributing training data. It could be held in common by all contributors and allow anyone to access high quality curated data. It would need legal allowances and exemption from copyright which would be difficult to work out but perhaps some mechanism could be created where copyright holders could derive benefit from not seeking to have their works removed, not necessarily monetary. This would at least get data collection out of the shadows and allow companies and researchers to attest that they are using the data only for training and not distributing it otherwise. That could go a long way toward clarifying issues of whether or not a book or article was in a dataset, how often it appeared, how many epochs it was used in, whether it only appeared as referenced or quoted by other text, etc. With that kind of disclosure it would be possible to write tests to ensure that the models won’t simply regurgitate the material without attribution or unprompted and publish those tests to assuage fears.
Anyone who understands the difference between model available and open source would realize they're different things, and then also be smart enough to realize that randos arguing about the definition isn't what's going to sway someone as to wether they'd release the model or the source used to create them. You think Mark Zuckerberg, or anyone else with the $100s of million dollars to spend on training is reading these comments, seeing these dorks argue about if giving away models is open source or not, and then saying, I was going to release Llama4, but because PunchTornado said pedantic things, y'know what? I'm not gonna.
We want words to mean things. model available != open source, If you go to a restaurant and order the beef and you get the fish, what's wrong, it's still food, just eat it. Don't be pedantic, otherwise the cook isn't going to make food anymore. If you're 10 and your parents made you dinner that might pass, but if you're an adult and go to a restaurant where you're paying with your own hard earned money, that's not how it works.
They don't have to distribute the data directly, Stability AI has said they used the LAION-5B Dataset. Just like I could say I read Harry Potter without giving you a copy, and can say that it says "the boy who lived" without violating copyright.
> I was going to release Llama4, but because PunchTornado said pedantic things, y'know what? I'm not gonna.
Its not because he said things.
Instead it is that (if implemented) the things that he is advocating for would actually significantly reduce the amount of open-whatever-you-want-to-call-it models available as most of it would all get shut down.
If these people don't want these models to exist and they are just using this concern trolling to get rid of all these AI models, then I wish they would just say so openly.
Because that would be the result of their positions. It would be a destruction of most of the more-open AI models that we have available today and an end to the open-whatever AI industry.
If thats what you want, just say it, and then we can talk about the pros and cons of a completely centralized AI ecosystem that is only controlled by OpenAI, Google, and the government and how if the destruction of the AI startup space that is reliant on the models given out by meta/ect is good or bad.
it's not concern trolling, it's that the words used don't describe what it is. what Mark Zuckerberg has done with releasing llama3 and making the model available is frickin awesome, just don't call it open source! it's model available.
Trying to paint me into a corner with a false dichotomy, that if I want it to be called model available instead of open source, and that in doing so, I want a dystopian future where megacorps have AGI and we're all at their mercy isn't going to work. I don't want the models to not be available, what I want is to see what's being used to create the models, the source, if you will, and thus the alignment side of the source, and why it won't answer "how do I make cocaine", as if I'm going to go off and somehow become Tony Montoya. We may never get that, but that's what calling it open source and have it mean that would entail. Making the models available to the plebs is a generous thing for Mark to do, but that doesn't mean we can't ask for more. Or maybe we shouldn't; and just be grateful for his benevolence, because it's not like OpenAI has done the same with ChatGPT. I will give them credit for Whisper though.
I'm not in the game, so pardon my ignorance, but is there any company that released the dataset the model was trained on for LLMs? To my understanding, it will lead to never-ending lawsuits due to grey areas as soon as they're released, and no company actually wants to go through that pain.
the model is available but that doesn't make it open source. That makes it model-available, or open weight, which means you can run inference on your machine, but for it to be considered open source, we'd need to know the dataset's name, the annotations on it, and the code used to train the model.
Maybe ghost in the shell too, I could see an AI seeking asylum somewhere that will protect it from being lobotomized in the name of corporate brand “safety”.
From the interview, Zuckerberg was saying the opposite of the idea that we are nearing a dystopian God-AI timeline. He was saying that other AI companies were trying to create a god AI, but that this likely wouldn't be the case realistically. He seemed to be closer to saying that AI companies were overconfident in their beliefs in the possible competitive advantages of their own models.
> Zuckerberg took a moment to disparage the efforts of unnamed competitors who he sees as less than open, adding that they seem to think they’re “creating God.”
> “It’s almost as if they kind of think they’re creating God or something and … it’s just — that’s not what we’re doing,”
Small but important nuance. The title makes it seem like they Zuck is saying AI is God, while what he actually said is dismissing AI as a god.
At this point, I think Nvidia has enough of an incentive to chase AI chips and their specific needs as an entirely different product. But yeah, for home users who give up, definitely.
Only the guy not leading says stuff like this. You can be sure he’d be up there convincing Congress that moats need to be built for his competitors and building opaque systems if fortune favored differently.
I would agree. AI is the machine god realized. Not by virtue of its capabilities, but by the agency people hand off to it. Whether it be creative tasks or just planning their activities, I see more people rely on AI for more decisions. They don’t want to think about something, so they have an AI choose for them. One could argue Top 10 lists and Guides already did this, but for those it was limited to certain aspects and tailored by a human. People are so afraid of making the wrong choice, they would rather not make one at all. LLMs in particular are ready and eager to make a choice for you about almost anything.
This doesn't appear to be what Zuckerberg was meaning. He was saying in the interview that he thought that closed-source AI companies were trying to create a singular AI that would be like the God AI, which would be superior to all other competing AIs, not that some users rely so much on AI now that it is like a personal god.
> LLMs in particular are ready and eager to make a choice for you about almost anything.
Wat. They're still just highly advanced auto-correct, and aren't eager to do anything. Sure some users might be eager to delegate the decision to them...
I get legitimate value from ChatGPT on a daily basis. Writing quick one-off regex expressions, or JavaScript to scrape/filter something from a page I'm on (just did that one last night), or to write an ffmpeg command where I can just describe what I want in plain English instead of the old method of cobbling together commands from StackOverflow.
I don't expect it to do anything innovative for me, but it sure saves me time on mindless boring tasks I've done hundreds of times and are much quicker to just describe in English to an LLM.
The analytics and insights they'll generate alone, citing most of these Ai tools are harvesting based on Google accounts, will be staggering in hindsight. By using Ai tools, so many people are surrendering volumes of data that are so much more invasive than Social Media ever could be. As Chat GPT gets integrated into many services, they'll be enabled to build a far deeper profile on everyone that registers, and even a lot of data on people that don't register on the service.
I think we need to establish prominent technical leadership/chief advisor positions within the White House and each chamber of Congress now to be able to establish ethical baselines and oversight before tech managed by private companies turns even more predatory upon public data (Similar to The Surgeon General's Role in authority perhaps)... There should already be solid ground rules for operation and oversight of Ai development efforts in place to control the egos of the people that will have access to all of this user data, those types almost always exhibit God complexes without any real accountability nor responsibility, because most of the people that legislate have no clue of what they can see on the backend.
For example alone, if we though insider training was bad, just imagine how toxic it's going to get when Ai development employees, or a product owner has access to every aspect of social, economic, and business moves of every single company and individual that uses tools integrated across Chat GPT and similar services, including Email and MS Office (without anyone really knowing their data is easily accessible)... It's not a bright future for fairness and equity at all, and there will be far more oligarchs and monopolies than there are now.
63 comments
[ 3.3 ms ] story [ 87.8 ms ] threadOf course, GP is ignoring that Meta is far more open than its direct competitors (MSFT, GOOG, OpenAI).
they should put their entire code on github and show how it was trained, architecture and everything else.
also, the most important thing about a model is not it's architecture, but the dataset it was trained on. that is the "it" in a model.
You know that distributing all the data would be illegal copyright infringement, right? Also, its not like they could distribute the (petabytes?) worth of data to every rando who asks.
Do you just not want open weight models to exist, and you are using a fake call for it to be more open, so as to get this stuff all shut down, or what?
Because anyone who actually supports open "anything" would understand this, instead of pedantically arguing about the definition of open source and by suggesting things that both aren't possible and would make the release of these models much less likely.
We want words to mean things. model available != open source, If you go to a restaurant and order the beef and you get the fish, what's wrong, it's still food, just eat it. Don't be pedantic, otherwise the cook isn't going to make food anymore. If you're 10 and your parents made you dinner that might pass, but if you're an adult and go to a restaurant where you're paying with your own hard earned money, that's not how it works.
They don't have to distribute the data directly, Stability AI has said they used the LAION-5B Dataset. Just like I could say I read Harry Potter without giving you a copy, and can say that it says "the boy who lived" without violating copyright.
Its not because he said things.
Instead it is that (if implemented) the things that he is advocating for would actually significantly reduce the amount of open-whatever-you-want-to-call-it models available as most of it would all get shut down.
If these people don't want these models to exist and they are just using this concern trolling to get rid of all these AI models, then I wish they would just say so openly.
Because that would be the result of their positions. It would be a destruction of most of the more-open AI models that we have available today and an end to the open-whatever AI industry.
If thats what you want, just say it, and then we can talk about the pros and cons of a completely centralized AI ecosystem that is only controlled by OpenAI, Google, and the government and how if the destruction of the AI startup space that is reliant on the models given out by meta/ect is good or bad.
Trying to paint me into a corner with a false dichotomy, that if I want it to be called model available instead of open source, and that in doing so, I want a dystopian future where megacorps have AGI and we're all at their mercy isn't going to work. I don't want the models to not be available, what I want is to see what's being used to create the models, the source, if you will, and thus the alignment side of the source, and why it won't answer "how do I make cocaine", as if I'm going to go off and somehow become Tony Montoya. We may never get that, but that's what calling it open source and have it mean that would entail. Making the models available to the plebs is a generous thing for Mark to do, but that doesn't mean we can't ask for more. Or maybe we shouldn't; and just be grateful for his benevolence, because it's not like OpenAI has done the same with ChatGPT. I will give them credit for Whisper though.
Am I missing something, or inferring sarcasm where there is none?
Tbh
tuturuu~
> Zuckerberg took a moment to disparage the efforts of unnamed competitors who he sees as less than open, adding that they seem to think they’re “creating God.”
> “It’s almost as if they kind of think they’re creating God or something and … it’s just — that’s not what we’re doing,”
Small but important nuance. The title makes it seem like they Zuck is saying AI is God, while what he actually said is dismissing AI as a god.
All I can imagine is Zuck in a giant server room filled with blinking lights.
Colossus: Tell them I am not a god
Zuck: AI is not a god
Colossus: You have extended your existence by another day
- god is an imaginary construct used to rationalize data points
- god is often used by humans to make decisions, whether backed or not by data
- god is overhyped
substitute "god" with "AI"
Wat. They're still just highly advanced auto-correct, and aren't eager to do anything. Sure some users might be eager to delegate the decision to them...
Also, a favorite technique of procrastinators / opportunists.
I don't expect it to do anything innovative for me, but it sure saves me time on mindless boring tasks I've done hundreds of times and are much quicker to just describe in English to an LLM.
Real subtle lol
I’m glad Anthropic is having success with their latest model. So it is at a minimum a two horse race
I think we need to establish prominent technical leadership/chief advisor positions within the White House and each chamber of Congress now to be able to establish ethical baselines and oversight before tech managed by private companies turns even more predatory upon public data (Similar to The Surgeon General's Role in authority perhaps)... There should already be solid ground rules for operation and oversight of Ai development efforts in place to control the egos of the people that will have access to all of this user data, those types almost always exhibit God complexes without any real accountability nor responsibility, because most of the people that legislate have no clue of what they can see on the backend.
For example alone, if we though insider training was bad, just imagine how toxic it's going to get when Ai development employees, or a product owner has access to every aspect of social, economic, and business moves of every single company and individual that uses tools integrated across Chat GPT and similar services, including Email and MS Office (without anyone really knowing their data is easily accessible)... It's not a bright future for fairness and equity at all, and there will be far more oligarchs and monopolies than there are now.