Ask HN: Should the US pass sweeping restrictions on AI development?
According to this government-commissioned report, AI at its current SOTA frontier represents an unprecedented existential risk to humanity. The best course of action, according to its authors, is for the United States to restrict compute on training models and punish open-sourcing weights with jail time.
> The new AI agency should require AI companies on the “frontier” of the industry to obtain government permission to train and deploy new models above a certain lower threshold, the report adds.
> Authorities should also “urgently” consider outlawing the publication of the “weights,” or inner workings, of powerful AI models, for example under open-source licenses, with violations possibly punishable by jail time, the report says.
> And the government should further tighten controls on the manufacture and export of AI chips, and channel federal funding toward “alignment” research that seeks to make advanced AI safer, it recommends
Do you believe, if the risks of AI are true, that this is the best course of action? Would this still allow AI development to reach a state of usefulness that has been promised for years, or will it completely destroy the momentum and lead to an AI winter?
54 comments
[ 3.1 ms ] story [ 136 ms ] threadI have not read the report, what are the reasons they recommend such drastic measures?
At this point, I think we can safely say that making a system that gives AI enough control to fuck up is a "skill issue" in earnest. The overwhelmingly obvious solution to this problem is to just not design systems that trust AI whatsoever. Problem solved, alignment crisis averted.
But anyway, there's no way to enforce not running certain programs without some kind of Orwellian surveillance baked in to hardware that I would abhor more than Roko's basilisk, and you now make the people at the forefront of the technology criminals and geopolitical adversaries. It would be an ultra-bonehead move. As always, technology and law don't mix well.
I feel like I can guess their stance on the topic without even knowing what it is.
Astounding. IMHO the biggest achievement wasn't technical at all--it was how they were all able to do this gigantic end-run around copywrite, trademark, or patent protections.
Essentially, there is no such thing as intellectual property anymore.
Could I host a Hollywood movie on my personal website? Could I take the text of "Harry Potter" and sell it as my own novel? If not, then intellectual property obviously still exists.
Inaccurate hyperbole doesn't help your side. It just makes claims about AI training overstepping IP boundaries seem illegitimate.
Why not? E.g., chagGpt 3.5, if you ask what the first paragraph of "Moby Dick" is, it prints it out verbatim. How is that not reselling somebody else's IP?
If you ask it to print out the entire book of "Green Eggs and Ham" it dutifully says it can't because of copywrite restrictions. HOWEVER, you can ask it to print out the nth page, and it will happily print it out, for all 1 <= n<= #pages in book.
It snorked up Dr. Suisses IP, and is reselling it to us.
> Could I take the text of "Harry Potter" and sell it as my own novel?
Well, I asked chatGtp to print out the first paragraph of Harry Potter, and it did, but said that it might violate copywright.... LLMs are quite "self aware" (for want of a better word) that they have appropriated copywrited IP. It will still sell it back to us.
LLMs are a compressed representation of the training set.
So...in answer to your question, no, you cannot take the text of "Harry Potter" and sell it as your own content. But chatGtp sure can!!!!!
> Inaccurate hyperbole doesn't help your side.
sigh yes, its important not to overstate your case. Let me try again: Intellectual Property Laws are for plebs like us. OpenAI and its cohorts, however, have found a way to rip off, repackage, and resell any IP they want to.
And all this fuss over how dangerous AI, and how it should be regulated, etc, is just a false flag to distract us from the real issues.
* Is the literal act of copying data to the hard drives used in model training fair use?
* Is the act of training using the data count as a performance of the work? (This one likely not because there's no human being performed to.)
* Are the resultant model weights considered a derivative work of some or all copyrighted works in the training set?
* Does a model reproducing some or all of a copyrighted work count as a performance? Does transmitting it as an API response? Does hosting it in the conversion history?
* If the answer to the above is yes who has done copying? OpenAI or the user? What situations is it one, the other, or both?
Nothing says you can't acquire data with reckless abandon, and use it to train your AI which would be fair use. I would love to see someone get sued for piracy and mount a defense that they were torrenting to gather data for AI training and see how it goes.
ChatGPT is a machine built and operated by humans. If ChatGPT infringes copyright, then it is those humans who own and operate it who are infringing.
The fact that they built a complex obfuscated system that stores many copyrighted works in a novel way (model weights) and reproduces them incompletely or unpredictably when asked, doesn’t change the fact that they have built a system whose purpose is to infringe copyright.
It’s a little bit like saying I started the car running and let it drive off on its own accord in front of a huge crowd of people, I’m not responsible for the deaths because it was the car that ran them over and the car doesn’t have agency. The car wasn’t designed to kill and never intended to kill. But it’s still me who operates the car and should have foreseen the obvious consequences.
I don’t buy into any of these arguments that just because ChatGPT is very complex or mysterious that somehow the humans who built or are running it aren’t responsible for its actions.
Interesting. Why do you feel as though this is the proper analogy, rather than being the car manufacturer who builds the car which could be used in this way?
What about a car manufacturer who knowingly does not put in a widget to ensure the car is never allowed to exceed the speed limit? Or does not install a breathalyser which drivers need to blow into in order to start the car?
It's also worth noting that simply reproducing copyrighted material is not necessarily an infringement of that copyright, and in most jurisdictions, it depends on the purpose the reproduction is intended to serve.
For example, the copyright on the book may have expired. Or it could be fair use (however that's defined in the user's jurisdiction). Or the user of ChatGPT may have a physical copy of the book and lives in a jurisdiction where this allows them to obtain a digital copy.
You are way overthinking it. Take my example of "Sam I Am" by Dr. Suess. I can buy access to chatGPT, and With the right prompts I can get it to print out every page of that book.
How is this different from somebody selling photocopies of the book in front of Barnes and Noble?
It doesn't matter what storage method or media is used. You can use a database, an LLM, translate it to swahili and back to english, whatever. If you can get somebody's IP without paying them, then axiomatically, indeed oxymoronically, their IP has been stolen.
If only
If AI means the freeing us all from living under the thumb of the current copyright cartel, I'd be dancing for joy. The tiny handful of powerful media conglomerates that sue 12 year olds for sharing a song, take down criticism by abusing automated DMCA systems for obvious fair use and use their monopolistic power to enrich themselves at the cost of everyone else in society should be dismantled.
Of course, in practice, I'm sure the lobbyists from the major copyright profiteers are already working to make sure they get unjustly paid and fair use for regular citizens gets further eroded.
It's dumbfounding to watch everyone equate piracy with hollywood movies, while the real innovators and scientists get thrown under the bus by theivery.
It is order of magnitudes more difficult to spend 10years studying something when your work can be stolen and reproduced at any moment.
The world is broken and ctrl c, ctrl v broke it.
Don't expect einstein level scientists and their downstream products that started the computer revolution... you killed the protections needed for the people that made it, cause you wanted movies for free.
It's insanity.
Intellectual property is a very weird construct. Imagine if we had Star Trek-style replicators, and in principle there was no scarcity of, say, diamond rings, but we passed a bunch of laws which forced you to pay $10,000 if you wanted a diamond ring.
But we do have Star Trek-style replicators for information!! So why do we take something which has no intrinsic scarcity, and impose this whole edifice copywrite, patents, etc, to artificially create scarcity, and make it property?
Because private property is how we keep score: you can get stuff you want if you make enough stuff that other people want. Market prices--how much you have to pay for stuff--is how we collectively decide what kinds of stuff other people want.
We have no other way of 1) signaling what kinds of stuff we want produced, or 2) motivating people to produce the stuff that everybody else's actually wants. Or, at least, it's the best way we've found to predict and plan what to do, without wasting effort and resources on stuff nobody wants.
We don't have to do it this way. Richard Stallman suggested a software tax: 2-3% of society are good programmers, so if we taxed society 2-3% of GDP, and distributed it to all good programmers, we could just let them work on whatever they wanted as long as they gave it away freely.
It's a great idea really. At least we wouldn't have the best computer science minds furiously working on how to make us doom scroll. But it's fair to ask, who decides who is a "good programmer?" How do we decide the appropriate tax rate, to ensure we're getting enough software? If programmers can work on whatever they want, how can we ensure we get the software we want and need? How do we allocate the money? Do we just distribute it evenly? Or do programmers get more proportional to (insert bogus metric here)?
We don't have any better way of answering those questions than by just creating a market for IP. And for a market to exist, there has to be scarcity. Treating information the same way you treat a plot of land or a case of beer seems really artificial, and it is absolutely artificial, but we don't know any better way to do it.
I think that ends up in a situation similar to The sciences - if you want to take on pay as a free programmer, not only does your work need to be open source, but it needs to take on a genuine need as defined by the rest of society. I imagine a market of sorts, where programmers are able to shop for valid "jobs". Jobs may even be defined by organizations of some sort. No jobs in this market are designed to help exploit users, and the generated code belongs to the legacy of all of humanity. You start getting paid once you start sending accepted PRS to established projects. And you can start your own projects and get them accredited.
It's not a perfect plan by any means - after all the sciences are already funded by about 4% of federal spending, and they use a similar accreditation process. But they too are struggling due to misaligned incentives and capital structures, such as publish or perish and the academic publishing industry. The resulting replicability and accessibility crisis is a serious inefficiency in turning science into a benefit for humanity.
It's quite hard to put the finger on the precise cause of all these misaligned incentives. But it is getting very very critical that we do. The ever increasing urgency of the matter will capture our best minds, I believe. I know it has guided my work as an open source commune director, open source contributor, open source mentor, and mathematician.
However, eventually, YouTube had to issue endless EULA-violations and copywrite takedowns...and the free ride was over.
Maybe if YouTube had convinced everybody that its tech was an existential threat, governments would still be debating about what regulations should be enacted, instead of enforcing IP laws on YouTube....
Just underscores what a GENIUS move it was for OpenAI to hype-up the existential threat angle. It got everybody scared, and when you are scared, your reasoning is impaired, and you start making dumb decisions. Just brilliant...
If intellectual property has the potential to threaten our survival, and I sincerely believe that it already is doing so, then we must dissolve every single piece of it, and make it a common good for all.
Even if China or some other country does not impose similar restrictions, a ban in the US and Great Britain would buy humanity a decade or maybe even 2 decades in which we might find a way out of the terrible situation we are in. (A decade or 2 is about how far the US and Great Britain are ahead in frontier AI research.) Also, buying an extra decade or 2 of life for everyone on Earth is a valuable goal in its own right even if after that Chinese AI researchers kill everyone.
If humanity can survive somehow for a few more centuries, our prospects start improving because in a few centuries, the smartest among us might be smart enough to create powerful AI that doesn't quickly kill us all. It is not impossible that the current generation of humans can do it, but the probability is low enough (3.5 percent would be my current estimate) that I'm going to do anything I can to slow down AI research or preferably stop it altogether -- preferably for at least a few centuries.
From what I read we have: massive unemployment, AI making military decisions. What else?
No one has a good idea on how to control an AI that is more capable than people are over a broad range of pursuits. (Actually we don't need to be able to control such an AI, it suffices for it to be aligned with our long-term interests. No one knows how to do that either.)
The Machine Intelligence Research Institute has been paying researchers to think up ideas for over 20 years. The executive director of that institute and its most senior researcher (Nate Soares and Eliezer Yudkowsky) have been publicly saying now for a few years that it is hopeless and that there is no way any of their institute's research directions will bear fruit before AI kills us all. They say that people just aren't smart enough to figure out how to control or align the powerful AIs that will be created soon if things keep on going the way they are going; consequently some sort of pause or halt on AI research -- almost certainly for at least a few decades -- is going to be necessary for us to survive.
The people in charge of the AI labs care enough about their own survival and the survival of their families that they could probably be persuaded to adopt an effective alignment method if someone or some group can come up with one, but it is highly unlikely that anyone will do so in time. Note that whether the effective alignment method is used to align the AI with the broad interests of the entire human population or whether it is used to align the AI to the particular interests of only the leaders of the AI lab is not a particularly important consideration at the current time: it only becomes important in the very unlikely event that someone somewhere finds an effective alignment method.
AI is essentially a subfield of the broader field of optimization; an AI applies an impressive amount of optimization pressure. (The human brain can, too.) When you (or a team of researchers) apply tons of optimization pressure, or create an AI that does so, you steer reality towards some "optimization point" or goal. Most optimization points are incompatible with continued human survival when enough optimization pressure is applied towards that point. And no one knows how to aim a powerful optimization process at the few points that are consistent with human survival and human flourishing. No one has even provided a mathematically precise specification of an optimization point consistent with human survival (and mathematical precision would be quite useful and I suspect also necessary for the specification to be useful to a team creating an AI). The closest anyone has gotten is a 2004 publication by Yudkowsky called "coherent extrapolated volition".
"The hidden complexity of wishes" is one way in which applying tremendous optimization pressure can go wrong, like in the story of the powerful genie who grants 3 wishes to the person (robber?) who found a magic lamp, but the hidden complexity of wishes is not the main snag.
If an AI is better at discovering scientific laws and inventing technologies than people are, it doesn't have to gain control over a nuclear arsenal: it can create its own arsenal that is better than nukes out of materials that people did not even suspect could be used as weapons (just as before about 1930 no one suspected that uranium could be turned into a bomb).
If we don't pause AI research till humanity is smarter, there is no 30% chance of success. If we look back at history at human projects that at the time were thought to have a 30% or 70% chance of success -- if we look back using our superior modern knowledge of those projects, you see that most were doomed from the start. Consider for example the project of turning base metal into gold. Completely doomed from the start. The alchemists living in the year 1450 did not know that: they thought they had a decent chance of succeeding. Consider most past medical treatments: cutting holes in a mentally-ill person's skull in the hope that that would let the bad humors leave the head; nailing religious symbols to front doors to prevent the house's occupants from catching the contagious disease going around. The thing all these doomed projects had in common is that the people pursuing the project lacked a sufficient understanding of the part of reality they were trying to affect.
Similarly, we don't understand how to align a superhuman AI. Some of us have been trying to understand it for 20 years. Some of us have been getting paid a salary for 20 years to try to understand it. No one has even gotten close. There's no 30% chance: there might be a 3% chance, and if you think life is so bad that choosing a lottery that ends humanity with p = 97% and removes human suffering with p = 3% is a good move, well, most people disagree with you. And (I mean this in all sincerity and kindness) I'm sorry life has been so unbearable in your experience.
It is better that the ban start now because no one has a method for determining which frontier models will become dangerously capable just as no one at OpenAI had any way of determining at the start of training that GPT-4 would be able to get a 92 on the bar exam. That is a pity because the benefits of creating more capable models is considerable: they could've made us all wealthier and saved lives through medical advances.
1. Bad people will use powerful models to do bad things.
2. Many jobs will be automated, and many people will get laid off.
2. This scenario I can understand causing _upheaval_ and significant societal change, but I can't see it causing the death of the human project like some predict. Will there be more food in this new society where it's incredibly cheap and efficient to make it? I presume so. Is it impossible for us to figure out how to distribute the food? No. Why will humanity come to it's end because people are not required to crunch numbers in excel day after day anymore? I'm not trying to be an ass, I just haven't seen an explanation that makes actual realistic conclusions.
1. Don't restrict this. Open source will develop AI further etc. Perhaps a knee jerk reaction at how one can use this tech in military applications.
Regulation to stop selfish profits, companies are lusting over themselves to make people's livelihood obsolete.
Capitalism and AI might be a toxic mix in the end.
- Power being consolidated to only some small number of companies that can lobby and spend well. This leads market capture and oligopoly. And bars smaller players from entering the market. This leads to poor conditions for consumers similar to aviation industry, cellular networks and broadband market.
- Innovation stops, and growth stagnates. Consumers and developers are left with poorer and higher paid products. The development of the technology stagnates.
- I guess government will also monitor development of weapons, etc. China, Russia won't have similar restrictions, and will get ahead.