There's a bit of a difference between producing the laptop that can connect to the internet and creating a web site with information on how to kill someone.
It is more just that there are a wide variety of existing ways available to harm or kill the population. No need to invoke the specter that AI magic is going to engineer some super bug. Plenty of virulent pathogens already around.
You could trivially cause enormous harm by sabotaging electrical distribution, water treatment, hospitals, airports, etc. Go to any food manufacturer/distributor and contaminate the products with listeria or such.
> Oh no, now this website has information on how to kill someone.
That's just a bad faith response.
> I'm not a bioweapons expert
That's the whole point, innit? You don't know how to do, so you can't. Publishing functioning recipes using easily accessed materials would provide you and countless others with a bio-weapon. Thus the publication is the problem, not the tech stack (internet, pcs, servers, etc.).
Therefore, OpenAI publishing risky information is equally at fault.
The same sneaky ploy used to restrict nuclear weapons to a mere handful of "great powers". Even today, most people believe that nuclear weapons are dangerous.
Unfortunately we have tried the destruction power of nuclear weapons on muck up cities and actual cities. This bold claim is not in agreement with experiment.
> danger of nuclear is not the explosion, that's nothing special, really
It's absolutely the explosion. (And fires.) You can make a similar boom as historical (not modern) nukes with conventional weapons, but not in a form factor that fits on the tip of an ICBM.
It's a fundraising move. OpenAI becomes concerned about how dangerous the things it's selling are when it comes time to fundraise. This nonsense doesn't work that well in America anymore, in part because our politicians bite the bait and then start regulating, which Altmen et al don't want, but you can absolutely get a term sheet in the Middle East and Asia by linking yourself to the national security budget.
No, prescient people were aware of these risks a long time ago, before any OpenAI company:
In 2022, a survey of AI researchers with a 17% response rate found that the majority believed there is a 10 percent or greater chance that human inability to control AI will cause an existential catastrophe.
Does your model of the world have anyone who genuinely believes that these things will someday develop levels of expertise in weapons (bio, cyber, or other) which, in the hands of terrorists or other malefactors, would be a real danger?
My model of the world does not recognize the possibility of a model that is simultaneously capable of designing new useful molecules while also not being able to create harmful ones.
Can't read the article because of the paywall, so this may/not be related...
Is it fair to say that the real liability here is a dataset mapping protein/molecule structures to outcomes/effects? Hypothetically the govt could always require OpenAI to blur responses with malicious intent. But if the underlying corpus is available, what's stopping a bad actor from training another model to do the same thing?
I guess the question I'm asking here is what risk is unique to their model if not the data it was trained on?
Depends what you're trying to do, light weight bio weapons are pretty easy to make, not an expert but there is ammonia and chlorine that can be used easily. I presume cytotoxins are among the easier of the "bad ones", but the precursors are monitored. that said, I think the risk here isn't about teaching people how to make the stuff we know about, it's about inventing new stuff we've not thought about?
Then you have to boot it up. Bio equivalent of putting the lightning in the rock (semiconductors). The DNA is like the lightning (like the code) the rock is the biological system you drop it into to run it. Booting viruses is not super easy.
Given how shit GPT is at programming and the amount of training data available in this domain - I highly doubt it would be more useful in this area over a Google search.
39 comments
[ 3.3 ms ] story [ 61.4 ms ] threadOh no, now this website has information on how to kill someone.
I'm not a bioweapons expert, but I think there's a lot of equipment and materials required to build a bioweapon of sufficient lethality and efficacy.
Look at anthrax for an example of a bioweapon with only one of those.
IDK much about Botox as a weapon, but can you use it to deploy broadly? With ricin, that's just another poison, so how's that better than rat poison?
You could trivially cause enormous harm by sabotaging electrical distribution, water treatment, hospitals, airports, etc. Go to any food manufacturer/distributor and contaminate the products with listeria or such.
That's just a bad faith response.
> I'm not a bioweapons expert
That's the whole point, innit? You don't know how to do, so you can't. Publishing functioning recipes using easily accessed materials would provide you and countless others with a bio-weapon. Thus the publication is the problem, not the tech stack (internet, pcs, servers, etc.).
Therefore, OpenAI publishing risky information is equally at fault.
Maybe you can't have such a big explosion with conventional weapons but you can just use more.
The problem is contamination.
Invisible and deadly
It's absolutely the explosion. (And fires.) You can make a similar boom as historical (not modern) nukes with conventional weapons, but not in a form factor that fits on the tip of an ICBM.
It's a fundraising move. OpenAI becomes concerned about how dangerous the things it's selling are when it comes time to fundraise. This nonsense doesn't work that well in America anymore, in part because our politicians bite the bait and then start regulating, which Altmen et al don't want, but you can absolutely get a term sheet in the Middle East and Asia by linking yourself to the national security budget.
In 2022, a survey of AI researchers with a 17% response rate found that the majority believed there is a 10 percent or greater chance that human inability to control AI will cause an existential catastrophe.
Altman, 2015: https://blog.samaltman.com/machine-intelligence-part-1
Bostrom 2014: https://www.goodreads.com/book/show/20527133-superintelligen...
Stephen Hawking, 2014: https://www.bbc.com/news/technology-30290540
Take your pick.
Are Geoff Hinton, Yoshua Bengio, and Douglas Hofstadter also marketing?
Government can try what it wants, we've done this before
Is it fair to say that the real liability here is a dataset mapping protein/molecule structures to outcomes/effects? Hypothetically the govt could always require OpenAI to blur responses with malicious intent. But if the underlying corpus is available, what's stopping a bad actor from training another model to do the same thing?
I guess the question I'm asking here is what risk is unique to their model if not the data it was trained on?
Source: my limited understanding.