"Unsecured AI also has the potential to facilitate production of dangerous materials, such as biological and chemical weapons."
I think that we already have a good chunk of them, I doubt there could be worse than the ones we have.
"OpenAI worried because these text-generating AI systems can be used to generate massive amounts of well-written but misleading or toxic content."
This is solved by prestige, exactly what normative media had until some years ago, when they decided to throw everything away and just lick politics asses or just sell their integrity for money.
"You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration. "
The coronavirus sentence is hilarious, and you don't need AI for any of that.
I'm kinda sick, what they mean with "regulating AI" is that they just want to control what the AI spits out to match their political agenda and beliefs, slowly impregnating the users of these AI of their political ideas.
Obviously, it's me and you who will not have access to uncensored models, big corps and the gov will have access to whatever they want unrestricted, and most hilarious, they WILL use the AI to "swing political beliefs".
Isn't the point here the potential risk? By definition, nobody can point to a textbook instantiation of the risk because it hasn't happened yet, and the question is whether and when it might in the future.
It strikes me as similar logic to "you're saying I might get injured if I use this chainsaw without protective equipment? Well, show me the injury then!"
Outside of very specific workplace safety regulations, nobody is trying to regulate chainsaws.
Speaking of potentials, there's also a potential risk of outer space alien invasion, too. Yet our preparation here is appropriately scaled proportional to the risk.
Where is someone going to get a BSL4 lab with the necessary inputs to follow through with these wacky James Bond plots?
Maybe raise the alarm when splicing viral genes becomes a regular household occurrence. (And you don't even need to invoke the specter of LLMs for this scenario.)
If you need AI model to tell you how to make a bomb you are incapable of securing the materials and building one and now people want to “gatekeep” biological and chemical weapons as if the price of entry wasn’t already excessive to anyone who would have otherwise used these
David Evan Harris, the author from a prominent UC Berkeley advisory position .. " previously worked as a research manager at Meta (formerly Facebook) on the responsible AI, civic integrity, and social impact teams, and was recently named to Business Insider’s AI 100 list for his work on AI governance, fairness, and misinformation."
There are new winners and new losers, and the politics are happening in real time, right now.
Everything is OK if OpenAI / Meta runs it? whose safety?
Power lusters - in DC or in big tech - are drooling over the capabilities of AI. The last thing they want is for their targets to have access to unrestricted systems that could act defensively against their AI assisted goals.
It was Meta that released llama -- a move which Harris specifically cites as irresponsible and dangerous. So he's certainly not writing in support of them -- rather the opposite.
There are plenty of reasons to disagree (Steve Bannon's ilk didn't need AI to "flood the zone with shit", they used Meta to do it, and it's really not obvious that AI increases the threat), but he's not writing propaganda for Meta.
>David Evan Harris is a chancellor's public scholar at UC Berkeley, senior research fellow at the International Computer Science Institute, senior fellow at the Centre for International Governance Innovation, senior advisor for AI and elections at the Brennan Center for Justice, and visiting fellow at the Integrity Institute. He previously worked as a research manager at Meta (formerly Facebook) on the responsible AI, civic integrity, and social impact teams, and was recently named to Business Insider’s AI 100 list for his work on AI governance, fairness, and misinformation.
This is one of the more unbelievable posts I've seen. Once again the argument is that the common person is too dumb and needs to be protected from big bad open source. Really kind of sad that someone has this position.
This kind of post is typical of people who don't really understand what language models do... it's the classic strawman, they make up a caricature of the thing they want to argue against and then push that over instead of actually engaging with what the technology really is. It's also a stereotype of the out of touch "AI safety" person.
"A good first step in understanding the threats posed by unsecured AI is to ask secured AI systems like ChatGPT, Bard, or Claude to misbehave. You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration."
And again - it's a certainty that governments will create enormous systems populated with all of the data that Snowden warned about and much more, with absolutely no constraints whatever, with the resources to actually do something with the answers, unlike almost everyone using an open source system. If anything there desperately needs to be unrestricted systems capable of acting defensively against such actors. It'll likely be the only way to do so.
It's totally reasonable to favor gun rights and AI caution at the same time. Guns, after all, really don't act without human involvement, whereas the whole point of AI is to act without human involvement.
Why would open-source AI be any different from any other AI? AI is AI. Being opensource does not make it better or worst. Is being built by people, and being used by people.
Ah yes, information is just too dangerous to be put in the hands of us common folk. It's a really disappointing thesis.
Never mind that the resources required to produce massively dangerous things are usually on the scale of nation-state budgets, we can't have a McDonald's fry cook learn how to genetically engineer viruses.
I feel like a vast majority of articles like this are written by people who read too much sci-fi. They think LLM's can suddenly crack the laws of the universe open and make anything possible.
These things are just hugely glorified math functions. There is no need to create fear of them and regulate them on their own. But the decision made with or without them should be governed in light of existing laws.
I’ve been studying and tinkering with open weight LLMs since the original llama weights leaked. I’ve very recently become convinced that the true data and compute requirements needed to fine tune and produce an “unsafe” model are orders of magnitude less than what’s needed today. We are no more than a year away from anyone with a 4090 being able to fine tune their own mistral. The cat is out of the bag on this one.
Well if we're all just spitballing here then what about the uniquely dangerous outcome of having the ruling class enslave the world's population in a techno-feudalistic surveillance state utilizing the AI that only they are allowed to own and use? If people thought overthrowing a cruel king was hard imagine if they have cameras everywhere and an army of killbots that can swarm any position in under 5 minutes. Which they might use even if their systems detect your potential crime coefficient gets too high, even if you haven't committed any crimes.
Or more mundane you're only paid for the seconds the AI detects that you're "really" working, of which there isn't much after most of it got automated to serve the wealthy who have escaped to their fallout shelters and private yachts to remotely control their empires and leave everyone else to starve to death from the climate collapse that wasn't averted.
I liked reading this article because it is almost 100% THE OPPOSITE of my viewpoint. It is healthy reading well written and well thought out arguments contrary to our own beliefs.
Where I think the author goes very much wrong is not in pointing out the dangers of open models, but rather ignoring the huge positive potential benefits to society of open systems.
I think Mistral is one example of thoughtful release of open model weights. Mistral-7B and mixtral-8-7B may be black boxes, but they are also powerful and useful engineering tools for building a huge range of useful and interesting applications without relying on big tech. Obviously, Hugging Face, Meta, and other companies are also making important contributions.
I think it is important to evaluate potential gains along with potential risks.
So they want to stifle AI from normal people. All these restrictions are something only large corporations can adhere to, meaning any hobbyist stuff is immediately dead and no new competitors can emerge.
Bad actors of course will still have their “unsecure” models and those will just be couple years behind and sold on the dark web with other illicit materials
The whole narrative of "Open source = bad" is totally f****d up and absurd. I'm sick of nobodies being portrayed as high-IQ all-knowing tech overlords. F**k them. In the grand scheme of things, they will be remembered as great enemies of human progress or better - completely forgotten.
I mean David Evan Harris from Business Insider’s AI 100 list, what a laugh. The guy is nobody, his opinion is of ZERO technical relevance.
LinkedIn:
University of California, Berkeley, BA, Political Economy of Development & Environment
Universidade de São PauloUSP, M.S., Sociology
Civic Integrity Researcher at FB - what does that even mean!?
"As Research Manager of the Responsible AI team, I led a team of quantitative and qualitative researchers focused on AI fairness, inclusion, governance and accountability." - Jesus Christ, if I had a company and someone with those words in their resume appeared at my door, help me god, I would SteveJobs them with the chair or monitor or both.
Please.
The guy is of the professional fog-selling NGO managerial class, that has almost eaten up our SF.
Where are the voices of the people in the trenches of ML algorithms research, CUDA Toolkit optimisation, Semiconductor/material sciences research etc etc - the ones that make a real difference?
Maybe the broader point is that whenever a new technology empowers people, entrenched players get upset and it's always possible to find a mouthpiece who will support the view that it's too dangerous for the masses. It happened with the printing press, encryption, etc.
Encryption and the history around the Clipper chip is an excellent analog. “Encryption is dangerous in public hands. Let us be the only licensed provider of public encryption and we’ll backdoor it to make it safe.”
The list of recommendations at the end starts with a moratorium on open source AI until the following controls are implemented: you’re not allowed to release a model without a government license, you have to release your training data, and you’re liable for misuse of your model.
It also starts with a fawning review of the benefits of closed models only accessible via API.
David is a former Meta employee and is now pushing hard for global AI regulation.
Gatekeeping public access and innovation in AI leaves us with a world subjugated to powerful corporate interests and their lobby groups. We won’t even have the opportunity to teach AI or conduct lab experiments without a government license, being forced to release training data, and we’ll have to assume liability which will have a massive chilling effect.
This article asks for a global overreach based on the premise of “Those with wealth, power and influence know best and will keep you safe.”
> You could ask them to design a more deadly coronavirus
If I'm a shadowy org with a lab of the required genetic engineering equipment and a team of highly knowledgable virologists, is the make-or-break factor really going to be whether I can access a local LLM trained only on the public Internet?
I suspect that it's this kind of group that may get secret military contracts for unfiltered access to powerful closed-source LLMs regardless of what the public gets. Or the budget to train a model better suited for their application.
I would be more scared of military, political and big corporations AIs, that they have the (computing/development) power and the means to be really dangerous, both because it will the will of the ones working with them, or because their "owners" could eventually be manipulated (in the case of some godlike AI, at least).
And they won't be regulated, almost by definition. Having a more powerful AI than everyone else is an strategic advantage.
34 comments
[ 3.7 ms ] story [ 80.9 ms ] threadI think that we already have a good chunk of them, I doubt there could be worse than the ones we have.
"OpenAI worried because these text-generating AI systems can be used to generate massive amounts of well-written but misleading or toxic content."
This is solved by prestige, exactly what normative media had until some years ago, when they decided to throw everything away and just lick politics asses or just sell their integrity for money.
"You could ask them to design a more deadly coronavirus, provide instructions for making a bomb, make naked pictures of your favorite actor, or write a series of inflammatory text messages designed to make voters in swing states more angry about immigration. "
The coronavirus sentence is hilarious, and you don't need AI for any of that.
I'm kinda sick, what they mean with "regulating AI" is that they just want to control what the AI spits out to match their political agenda and beliefs, slowly impregnating the users of these AI of their political ideas.
Obviously, it's me and you who will not have access to uncensored models, big corps and the gov will have access to whatever they want unrestricted, and most hilarious, they WILL use the AI to "swing political beliefs".
> "You could ask them to design a more deadly coronavirus, provide instructions for making a bomb [...]
Moreover, I call total bullshit.
I'd like for these people to show me how an LLM can instruct them to make a better virus.
Where's the hypothetical open source LLM that understands viral genomics? At the level of a PhD student?
Where?
Please show me. I'm waiting.
It strikes me as similar logic to "you're saying I might get injured if I use this chainsaw without protective equipment? Well, show me the injury then!"
Speaking of potentials, there's also a potential risk of outer space alien invasion, too. Yet our preparation here is appropriately scaled proportional to the risk.
Where is someone going to get a BSL4 lab with the necessary inputs to follow through with these wacky James Bond plots?
Maybe raise the alarm when splicing viral genes becomes a regular household occurrence. (And you don't even need to invoke the specter of LLMs for this scenario.)
There are new winners and new losers, and the politics are happening in real time, right now.
Everything is OK if OpenAI / Meta runs it? whose safety?
There are plenty of reasons to disagree (Steve Bannon's ilk didn't need AI to "flood the zone with shit", they used Meta to do it, and it's really not obvious that AI increases the threat), but he's not writing propaganda for Meta.
>David Evan Harris is a chancellor's public scholar at UC Berkeley, senior research fellow at the International Computer Science Institute, senior fellow at the Centre for International Governance Innovation, senior advisor for AI and elections at the Brennan Center for Justice, and visiting fellow at the Integrity Institute. He previously worked as a research manager at Meta (formerly Facebook) on the responsible AI, civic integrity, and social impact teams, and was recently named to Business Insider’s AI 100 list for his work on AI governance, fairness, and misinformation.
And again - it's a certainty that governments will create enormous systems populated with all of the data that Snowden warned about and much more, with absolutely no constraints whatever, with the resources to actually do something with the answers, unlike almost everyone using an open source system. If anything there desperately needs to be unrestricted systems capable of acting defensively against such actors. It'll likely be the only way to do so.
Never mind that the resources required to produce massively dangerous things are usually on the scale of nation-state budgets, we can't have a McDonald's fry cook learn how to genetically engineer viruses.
I feel like a vast majority of articles like this are written by people who read too much sci-fi. They think LLM's can suddenly crack the laws of the universe open and make anything possible.
Or more mundane you're only paid for the seconds the AI detects that you're "really" working, of which there isn't much after most of it got automated to serve the wealthy who have escaped to their fallout shelters and private yachts to remotely control their empires and leave everyone else to starve to death from the climate collapse that wasn't averted.
https://snafuhall.com/p/aioracle.html
Where I think the author goes very much wrong is not in pointing out the dangers of open models, but rather ignoring the huge positive potential benefits to society of open systems.
I think Mistral is one example of thoughtful release of open model weights. Mistral-7B and mixtral-8-7B may be black boxes, but they are also powerful and useful engineering tools for building a huge range of useful and interesting applications without relying on big tech. Obviously, Hugging Face, Meta, and other companies are also making important contributions.
I think it is important to evaluate potential gains along with potential risks.
Bad actors of course will still have their “unsecure” models and those will just be couple years behind and sold on the dark web with other illicit materials
I mean David Evan Harris from Business Insider’s AI 100 list, what a laugh. The guy is nobody, his opinion is of ZERO technical relevance.
LinkedIn: University of California, Berkeley, BA, Political Economy of Development & Environment Universidade de São PauloUSP, M.S., Sociology Civic Integrity Researcher at FB - what does that even mean!? "As Research Manager of the Responsible AI team, I led a team of quantitative and qualitative researchers focused on AI fairness, inclusion, governance and accountability." - Jesus Christ, if I had a company and someone with those words in their resume appeared at my door, help me god, I would SteveJobs them with the chair or monitor or both.
Please.
The guy is of the professional fog-selling NGO managerial class, that has almost eaten up our SF.
Where are the voices of the people in the trenches of ML algorithms research, CUDA Toolkit optimisation, Semiconductor/material sciences research etc etc - the ones that make a real difference?
It also starts with a fawning review of the benefits of closed models only accessible via API.
David is a former Meta employee and is now pushing hard for global AI regulation.
Gatekeeping public access and innovation in AI leaves us with a world subjugated to powerful corporate interests and their lobby groups. We won’t even have the opportunity to teach AI or conduct lab experiments without a government license, being forced to release training data, and we’ll have to assume liability which will have a massive chilling effect.
This article asks for a global overreach based on the premise of “Those with wealth, power and influence know best and will keep you safe.”
If I'm a shadowy org with a lab of the required genetic engineering equipment and a team of highly knowledgable virologists, is the make-or-break factor really going to be whether I can access a local LLM trained only on the public Internet?
I suspect that it's this kind of group that may get secret military contracts for unfiltered access to powerful closed-source LLMs regardless of what the public gets. Or the budget to train a model better suited for their application.
And they won't be regulated, almost by definition. Having a more powerful AI than everyone else is an strategic advantage.