It is not one angry person shutting down the internet or a massive supply chain attack that worries most as a security researcher. I have come to accept such things are way too easy, and therefore likely, but as a species we will survive without Instagram.
But what actually worries me most is biosecurity.
Someone angry at the world with the $800 of CRISPR gene editing equipment at home and an AI model coaching them through engineering a virus for maximum harm.
They will absolutely not create anything of note with that setup, engineering viruses de novo requires a properly stocked lab with quite a few more zeros at the end of your proposed figure. Guess what happens if you try to assemble all of that as some rando?
The genetic sequence for smallpox is publicly available. The bottleneck in causing maximum harm is that you need a large, expensive, well stocked lab and a lot of expertise. LLMs helping people generate a sequence just isn't the bottleneck to bioterrorism, it's entirely about abilities and equipment to deal with the physical world.
It's still worth thinking about it, of course, but by far the way to mitigate risks are things like wastewater surveillance, tracing purchases of physical supplies, etc...
Of course, an LLM could provide advice that makes it easier for a virologist to do something like this, but there is little indication the effect size there is notable.
The life sciences are very hard. A motivated undergrad requires extensive in-person teaching to become useful in far simpler projects. It's not clear how an LLM can in the foreseeable future bridge that gap when human experts can't teach someone without a lot of time and in person interaction, and that is with access to a staffed, resourced lab.
This is an absurdly irresponsible way to report this work. Doesn't think to mention that they're working with phage (viruses that infect bacteria, NOT humans) until halfway down the story.
8 comments
[ 1.5 ms ] story [ 27.0 ms ] threadBut what actually worries me most is biosecurity.
Someone angry at the world with the $800 of CRISPR gene editing equipment at home and an AI model coaching them through engineering a virus for maximum harm.
1 angry person with $800 in CRISPR and AI
1 angry person with $800 in pangolin cages and time
1 company with a billion chickens and a drive to cut biosafety costs
It's still worth thinking about it, of course, but by far the way to mitigate risks are things like wastewater surveillance, tracing purchases of physical supplies, etc...
Of course, an LLM could provide advice that makes it easier for a virologist to do something like this, but there is little indication the effect size there is notable.
The life sciences are very hard. A motivated undergrad requires extensive in-person teaching to become useful in far simpler projects. It's not clear how an LLM can in the foreseeable future bridge that gap when human experts can't teach someone without a lot of time and in person interaction, and that is with access to a staffed, resourced lab.
Your concerns are grievously undermined.