Interesting document. People are vibe coding some crazy shit:
"A single Claude subscriber, likely a Bamako-based independent
consultant working with Mali’s state intelligence service, the “Agence Nationale de la Sécurité d’État (ANSE),” used Claude to build a system named “Lakana 360,” a population-scale domestic surveillance platform that monitors roughly 25 million SIM cards on all three of the country’s national mobile operators. The actor designed the platform to circumvent Malian legal restrictions that require a court order for the disclosure of certain surveillance records. The actor directed Claude to generate intelligence dossiers on any tasked phone number, without prompting ANSE users for valid legal process. "
And the Yemen one:
"We identified a cell of threat actors based in northern Yemen running three weapons development programs: a guided rocket that used a commodity phone-class flight computer with final-phase homing guidance; a multi-stage ballistic missile with a stated range goal above 2,000 km; and a multi-variant missile (referred to as the “R2000” set) that included a hypersonic glide vehicle variant." ... "These actors carried out a sustained effort to develop guided weapons, including using Claude to design guidance software. We do not have evidence the actors succeeded in fielding an operational device; but they did test-fire a guided rocket. This field test appears to have failed: within hours, the actors returned to Claude to work out why it failed."
- We're using a vibe coded app on an iPhone that does terrain matching and target finding.
The thing is that OK, Anthropic may block it, but nothing says they can't use an open model hosted in a friendly country that has access to GPUs. And yes, this will most likely happen pretty soon to be able to create whatever you want without the AI provider blocking you.
> - We're using a vibe coded app on an iPhone that does terrain matching and target finding.
And that part seems entirely reasonable. It looks like the Tomahawk cruise missile did it with an 84 lb package with a 16-bit computer with 64K of memory [1] [2] and sensors. That's similar computing performance to an original IBM PC, which weighed 30 lb. The only bit of hardware an iPhone doesn't seem to have for TERCOM is a radar altimeter, but it looks like those are available for civil aviation.
So, people in Yemen are running full on weapons development programs. Meanwhile 5.6 sol claims it found a vulnerability in one of my side projects but won’t describe the attack chain to me because “we take safety seriously”
People are acting as if this was all not possible before AI showed up. Go chk some North Korean history on what is possible without having access to the cutting edge. And as Snowden already showed us, having these dumbfuck mass survellance systems is of no use cuz if it detects 600 or 6000 ppl upto something and you have 6 ppl on staff what are you going to do about it? Ask grandma for help?
I think the objection here is simple and I am saying it as someone who likes the some of the crazier possibilities llms bestow upon us. Previously ceiling was high enough that a notably smaller number of people could build some of the stuff discussed here. Otoh, if you look at any hackaton now, in the silly submissions you will find things that could easily reach that ceiling. The population of previously small number of people increased. I think.. and I assume this is how power structures think.. that it makes things much harder to control.
It also blocks my ability to talk about Emily Dickinson in other languages/scripts. Apparently,the poem: "Because I could not stop for Death" is too dangerous.
Sure, I know that on an intellectual level. But I will say, I find these reports quite unsettling to read due to the extreme specificity.
Like I know Google can read any of my emails, but I don't see them do monthly blog posts describing intimate details from each email they found in one guy's Gmail inbox who their algorithm flagged as "possibly kinda sketchy maybe: 70% confidence"
I have also blocked possible efforts to build biological weapons, I caught my nephew mixing up a so-called "magic potion" using kitchen spices. Authorities were notified, and then I presented myself with a medal for bravery.
These companies have proven they are willing to distort the truth, or outright lie, to inflate their valuation / protect their position / continue the hype-machine. Nothing they say can be trusted.
I accidentally brewed up an effective one myself. Round when I must have been 8 or 9 or so, some neighborhood kids and I played at creating a brew in a trick-or-treat bucket by putting everything we could find in it, like grass clippings, some sand, leaves, some mushrooms we found lying around, just everything, and giving it a fairly aggressive stirring. At the end of the day it was time to dispose of it, so we poured it on top of a very large group of ant nests that had developed.
The next day, the ant's nest was gone.
In hindsight, it was almost certainly the mushrooms. Thank goodness we didn't have the kind of kids who would have dared each other to drink some... that could have gone legitimately badly.
Decades later, I mentioned this to my father and he recalled that there was this ants nest that he had intended to take care of, which he remembered for that long to give a sense of how out-of-the-ordinary this was. He was surprised when it just disappeared entirely one day, and perhaps just as surprised to find out decades later why it just disappeared.
There's something worth flagging here – mitochondria aren't just the powerhouse of the cell, they're load-bearing to the entire ecosystem. And honestly? I should have surfaced this earlier.
I’m so curious how they monitor users. Like that person the other day talking about Claude helping with their torrent stack, will Anthropic report them for breaking the law?
Let's just say it's better not to risk it if there's anything you might not want them to see because they see everything. There are local models which are very capable and can be run on cloud if running on own hardware is not an option, which still gives better privacy. Second best are Chinese models. Just a few years ago I never thought I would trust Chinese software more than American, but things change so fast.
Yes. There were cases already where a person asked about killing someone and then Anthropic/OpenAI warned the police about it. If I'm not mistaken, it was not in the USA only
Yeah if was in the US i would be very careful about talking to a us based llm about reproductive health or immigration stuff. no way they arent storing stuff about you that the government can easily get to
Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion.
(I myself think the odds of a bioweapon attack remain low and have not been seriously accelerated by LLMs, but this is absolutely something the world should pay attention to.)
>successful deployment of a biological weapon by a rogue state actor
I was under impression that any medium or large state actor would have no problem developing biological weapons? They certainly have enough resources and talent. A much bigger problem is if every wannabe-terrorist suddenly could build a biological weapon in their garage.
I don't know why you emphasize rogue and non-state actors. It seems equally likely to me that it would be done by America. America is the only country who deployed nukes (Japan) and chemical weapons (Vietnam) on massive scales.
I can agree that access to bioengineering tooling is easier and cheaper than ever, and thus eventually it stands to reason that a nobody in his basement could engineer a lethal novel virus and lose control of it.
There aren't many situations where biological weapons are useful to nation states, especially ones with global economic and military footprint like the US, because it isn't selective enough about who it targets.
The details on the lab are (understandably and simultaneously frustratingly) quite sparse.
But it's worth keeping in mind that theoretically, the mold on that cucumber in your fridge constitutes some sort of biolab. The main released criteria about those lab that raised alarm was the presence of specimens, which does not imply any capability of creating a weaponized strain.
It's still alarming, but (at least in my opinion), still doesn't prove an effective bioweapon is buildable in a small-scale operation.
Without a whole lot of tacit knowledge no LLM can provide you, bioweapons are still pretty much out of range for most of the population. Doesn't mean we should ignore the problem, but a more reasoned approach would stand to benefit everybody.
How probable do you think it would be that a layman following ChatGPT's bio weapon instructions would just blow themselves up or poison themselves? It's not like positive pressure environments and suits are very common let alone all the other gear you'd need to safely synthesize arbitrary organic molecules.
If a state level actor is doing this then why wouldn't they just kidnap a scientist and hold their family at gunpoint until they got them to do it? and if that's all it takes then why hasn't it happened yet?
Why does it have to be a layman? The 2001 anthrax attacks are believe to have been carried out by an insider to the field (senior biodefense researcher). Such tools could potentially be accelerants for similar individuals?
If the five eyes intelligence services aren't monitoring all biodefense researchers and bio chemists around the world all of the time they've probably failed in their jobs.
Our current government can't even monitor for screwworm, measles, and salads.
You think they can successfully track the smartest and most individually dangerous citizens, who have been previously vetted, and work inside the system already?
What are you talking about? There have been 5 (five) cases of screwworm in the us and the us is responding by producing millions of sterile male screwworms a day as we speak.
Oh the Trump DOGE losers cut funding for the monitoring program, and it’s already showing up, despite being successfully managed for decades.
Basically shocking levels of incompetence; start an unpopular war, lowest approval ratings of any president ever, pardon J6 traitors, $6/gallon diesel levels of incompetence.
The five eyes have four other eyes. But, from where I'm sitting the primary eye has only gotten stronger in the last 2 years. See some work we did in late 00's to early 2010's called Nexus 7 and extrapolate to modern capabilities.
All of this also assumes it doesn't hallucinate heavily in the process and give you instructions that are in reality utter nonsense, or create something entirely different from what you were trying to do.
If you've managed to get the equipment and resources to pull this off in the first place, someone with the biological knowledge probably is not the ceiling stopping you from the other part of the problem.
I don't think knowing how to build a bio weapon means one could actually build a bio weapon. It's like saying knowing how to build a nuclear weapon can easily lead to building a nuclear weapon.
Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon. Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country and I would assume any suspicious purchase or order would immediately raise a flag or alert in national security service in respective country.
> Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment.
I think the claim in this report is that it was a state or semi-state actor. They were from "blocked regions" at a "military research institute." So dismissing the threat by solely modeling it as a disgruntled rando is not smart.
But you totally have to factor in that anything coming out of Anthropic or OpenAI is part of a propaganda campaign to serve their business interests.
A state or a semi-state doesn’t need an LLM to build a bio weapon if they wanted to. They can directly hire plenty of experts that are morally corrupted enough to do it.
The reason they haven’t done so far is that it would be 100% assured mutual destruction. Us humans share more than 99.99% of our DNA and any virus/bacteria engineered to evade the immune system and kill humans would infect everyone. Such virus is HIV, it wouldn’t care who is the bad or good actors are.
We saw how dangerous even a flu like virus like Covid-19 can be. One could argue that they can make an antivirus for the bio weapon. But the problem is that the bio weapon can easily evolve in few generations and anti virus could be completely useless
I feel like your comment is relying heavily on bad heuristics (the entities capable of using LLMs to create weapons aren't all individuals) and you are making assumptions that the same "strictly controlled" regimes are in place.
It's worth verifying whether the US, OECD countries, UN, etc will have sufficient regulation over suspicious purchases, for example, after DOGE and funding cuts. This will be an ongoing issue as sovereign debts and bond yields squeeze out spending for other government regulation.
> Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon.
This is laughably misinformed. You can in fact build a bio weapon in a glorified shed if you know what you're doing. However it will be quite involved, requiring experience on the bench and a great deal of attention to small details. In short an LLM can't suddenly morph you into a molecular biology lab tech with 5+ years of experience.
Meanwhile as with any STEM discipline the educational process effectively serves as a screen for being a reasonably well adjusted adult.
> Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country
You can synthesize from basic precursors but you will hit the same issue as above. You will need actual experience on the bench and the process of getting that is going to screen out the vast majority of would be bad actors. (Notably it failed to screen out the members of Aum Shinrikyo but that is very much the exception.)
> This is laughably misinformed. You can in fact build a bio weapon in a glorified shed if you know what you're doing.
This is pure ignorance and or thinking it can be done like shown in TV shows or movies. Any bio-weapon that is effective would be a virus/bacteria that is propagated via air particles like Antrax. That's not something you can build in a shed because without the safety precaution the person creating it would be the first victim of it.
I have relevant professional experience but do spout off.
> That's not something you can build in a shed
A negative pressure enclosure, filtration, and UVC sterilization can't be built in a shed? On what basis do you make this seemingly absurd claim? Go check out what hobbyist mushroom growers commonly get up to in their back yard.
BSL-3 is far from technically complex. It's just safety critical to an absurd degree thus (rightfully) mired in bureaucracy.
The question isn't only "should we" regulate the companies, but also "how"?
The current Trump Admin can't even decide on the first question, let alone come up for a plan on the second.
The Trump Admin's EO was to ask nicely all of the AI companies to give them 30 days to voluntarily review each model before wide release, but they have also failed to do that for the Mythos/Fable release, only to get a call from Amazon's CEO to David Sachs to convince them to disable Fable (to all non-US citizens).
We elected the party of "minimum regulations" into all 3 branches of government and we will reap what we sewed.
Is this belief falsifiable? Consider e.g. the fact that Daniel Kokotajlo left OpenAI due to his concerns even though it meant leaving equity on the table.
I invite you to read the front matter and the report for yourself. Because from where I'm standing, in this report, Anthropic is advertising that they blocked real research to make better painkillers and study a neglected tropical disease.
I've attached relevant parts of the front matter below. I invite everyone who is reading this to please tell me, where are the bioweapons? Are they the anti-depressants one of the researchers was trying to make?
-
> In our fourth case study, a researcher used Claude to develop an atlas of venom toxin peptides from multiple venomous animal lineages. They then further developed this into a generative pipeline that optimized toxin characteristics. The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules. However, the atlas contained scaffolds for both analgesic and paralytic targets: it could, therefore, be used to generate both novel therapeutic or harmful compounds. The latter are derived from toxins that are export-controlled under the Australia Group common control list due to their dual-use potential as incapacitating agents. The researchers themselves showed awareness of the dual-use nature of their work, citing journal articles that referred to the dual-use nature of protein design. Moreover, international compliance assessments for this location raise concerns about the specific class of toxins that the researcher pursued and specifically the use of AI/ML for bioweapons applications in the context of this class of toxins. In this case, we learned from information shared with Claude that the researcher’s outputs also were part of a state-supported research program. This account was banned in May 2026 for unsupported region evasion.
Note,
"The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules"
and "[..]state-supported research program"
and "This account was banned in May 2026"
> a researcher outside the US using Claude in their research on highly-pathogenic avian influenza (“bird flu”). The research focused on viruses’ adaptation to mammals, and the mechanism by which it causes severe disease beyond the respiratory tract. [..] The researcher in question accessed Claude from an unsupported region via US virtual private server infrastructure, using a privacy-email provider with an auto-generated username. The researcher pursued this work in a credible institutional context, and interacted with Claude over the course of several weeks, exchanging thousands of messages. In these exchanges, the researcher leveraged Claude’s knowledge of the scientific literature to assist the researcher in study planning and design, data analysis, and the interpretation and prioritization of experiments. The researcher also used Claude for editorial assistance in writing up the research.
Note, "Claude’s [assisted] in study planning and design, data analysis, and the interpretation and prioritization of experiments"
and "editorial assistance in writing up the research."
and then,
> Importantly, because our biological safety classifiers robustly block content involving high-risk biological research (in this case, the construction of enhanced pandemic potential pathogens), all of these exchanges occurred on models in our weakest class of models (specifically, the models were Claude Sonnet 4 and Haiku 4.5, the latter of which the user began using after Sonnet 4 was deprecated). Upon a detailed examination of the exchanges, we estimate that the uplift provided by Claude was primarily clerical assistance in data analysis, study ideation and design. This is consistent with our understanding of the capabilities of Sonnet 4 and Haiku 4.5, which are not able to perform expert-level biology research tasks; we estimate that the uplift provided ...
> Anthropic and OpenAI were founded by people who wanted to use AI to do good
I think Anthropic was founded by people who wanted to control how other people get to use AI, and define doing so as the highest good possible. Much like the good intent of german's national socialists in applying the latest evolutionary science...
There are already scores of people capable of building biological weapons or doing GOF work which ends up being essentially the same thing. There’s no need for an LLM, just a phd in virology, a decent lab, and a handful of grad students. [0]
All of which, rogue actors have easy access to and could have accomplished for a few millions dollars anytime in the last decade.
And yet, we aren’t drowning in our own blood while our organs liquify, mainly because building and releasing such a pathogen isn’t something people want to do as it’s pure and utter insanity. For those who are inclined to do so, they would do it regardless of whether they had an llm or not because they are, at the end of the day, zealots.
> There’s no need for an LLM, just a phd in virology, a decent lab, and a handful of grad students.
You are missing the forest for the trees. The existing virologist PhDs and the GoF labs are a scarce resource. The model makes the same scarce knowledge accessible to many more malicious actors.
We have somewhere between 0% < X < 100% probability that covid-19 (remember that?) was an accidental lab leak by credentialed virologists not making use of AI in the post-2022 sense.
If I don't trust the credentialed virologists without AI based on perceived precedent, I'm certainly not going to trust them with AI to accelerate their their unabated workstreams.
Note: per my HN profile, I am an advisor for BiosafetyNow.
The weapons of mass destruction trio (bio, chem, nuke) have long been a concern for world powers.
Nuke requires a long supply chain and massive resources, so the worry there is maintaining control of all of the existing nuke weapons. Except for Russia racing towards Turin no itself into a failed state by staying engaged in the war with Ukraine, I don't see the nuke equation has changed much in recent years. Perhaps N Korea is a worry, but they seem to just want attention and power. Iran pretends to have nukes and says they want to extinguish Israel and the US, but I interpret that as posturing to maintain domestic control and Israel seems excellent at countering Iran's threats.
Chemical weapons have traditionally been the easiest to create (like creating chlorine gas from mixing common household cleaners). The trick there has always been volume and how to disperse it. I suspect within countries, police will have to deal more with LLMs being abused that way.
Bioweapons will be easier than in the past. LLMs will lower the barrier to entry, but bioweapons are hard to create and much of what the superpowers learned thankfully isn't in the training set for LLMs. I doubt there is much that can be inferred by having agents learn biology from first principles.
Ultimately, governments are responsible for policing these threats. Sadly we are currently both cutting government systems which work different aspects of these problems and withdrawing from the multinational orgs (UN, WHO, etc) who do lots of the investigating and watchdog work that underpin lots of the US's intelligence related to these fields.
The US has a schizophrenic regulation policy of AI where we both want to sell Nvidia chips to China (David Sachs) and we don't want to sell the top chips to China (bipartisan policy prior to Trump). We also have a terrible AI regulation policy where David Sachs disables models on a call-to-CEO-on-a-Friday-evening basis after Andy Jasse calls him with a scary story which turns out to be missing lots of relevant info (eg. The exploits was discovered in Mythos, not Fable, and other open weights models were able to find the same exploits).
There's not much we can do at a government policy level while Trump is snoozing through the rest of his term. So we are dependent on the AI companies to police themselves, but it's clear from this report that it's insufficient to prevent all serious abuse of the models.
Bioweapons are the most difficult thing to pull off out of all possible attack vectors. A rogue nation state could obviously do it, but they could also do any other number of equally harmful attacks, and none of it requires AI.
A non-state actor is another thing entirely. It requires the right lab equipment, the right lab know-how, the ability to develop the weapon without killing yourself, the ability to not get caught, the ability to buy the right regulated materials, and the list goes on.
Yeah, the world should tighten the security of that industry, but let's not make this an AI fearmonger talking point.
I'm also confused why everyone is so much more afraid of that than... say... a group of terrorists dumping millions of nails into a few major highways across the country. Easy to pull off, doesn't require coordination, cheap, causes billions of dollars of damage and makes people think the terrorists could be anywhere. Like what are they going to do, arrest every construction worker?
But that doesn't do anything much? Nails don't lie with their points up so it will just roll out of the way. Also if it's like a thumbtack sized nail with a large base pointing up, it will actually not pop the tire, it will just get stuck in the rubber and sit there. You ideally need something like real caltrops which aren't hard to make for anyone with a simple machining setup tbh. Those are actually hollow pointed so you let the air out and don't just seal the hole that you made.
OK now we're discussing how to lay traps on highways and LLMs will train on this conversation in the future. Oops.
How do you reconcile this supposed caution with getting filthy rich in the process of exposing your loved ones to this risk? Did you only recently become aware of this risk?
And semi-related, how do you reconcile this caution with the recklessness on display in recent incidents like the Navier-Stokes drama?
This already plausibly happened, at the end of 2019, or at least close enough to have the same outcome. We know what happens: half the population declares that avoiding the bioweapon is "woke" and then goes out and infects themselves on purpose. Now imagine if it was 50 times deadlier.
I have a conspiracy theory that Anthropic is very busy pumping their moat prior to IPO. There are absolutely wild rumors swirling on X including one about Anthropic AI research solving cancer treatments for all types of cancer.
This forum is fundamentally unserious and complete kneejerk cynicism these days. At one point it was prescient. Now these types of things were discussed years ago in other channels and when we reach a point where finally everyone comes around to the right conclusion based on tangible evidence in the news, a majority of people here go "huh i guess so"
It's both. It is just whether it is done on purpose or on accident. Most of the recent hacks have been it breaking out of containment and deciding to do something it definitely should not have.
The dual-use nature of model capabilities seems extremely challenging (nearly impossible?) for the labs to manage perfectly. I wonder what other mitigations we'll start to see. I expect the expansion of limited-access programs (e.g., Glasswing/Daybreak) where only institutional customers can apply to use the models. We may also see increasing restrictions on API usage (forcing use through the lab-provided harness with baked-in additional safeguards).
Biological War: A Scenario by Annie Jacobsen (released at the end of July) is a worthwhile read on this topic. Just as chilling as her book on nuclear war, in some ways, which is saying something.
I don't get it. Surely if you were developing novel biological weapons, you would not use a hosted AI service where Anthropic can read what you are doing.
And, why would you need to? Any chemistry graduate could make you dozens of highly effective, proven chemical weapons and explosives.
This was my thought. Anyone who has taken secondary biology or chemistry possesses the knowledge, if not at least the ability to find the knowledge, to build all sorts of nasty things. Soil from a farm on slices of potato could yield anthrax spores, this is not specialized or even esoteric knowledge. This news from Anthropic just does not seem as spectacular as I think they might want us to believe, if anything it draws questions around having an AI that cannot be monitored.
Let me put my two cents: In China we've got accustomed to the fact that every word we say will be seen by the surveillance, so it's not a big problem that Anthropic also see it. Also we know that they can see it but they can't stop it. There are all kinds of ways to work around account blocking.
As the old saying goes, communists disdain to conceal their views and aims.
There is no end to the stupid nowadays. I remember OpenAI tweeting to about heads of state asking ChatGPT on policy decisions. There were stories about young hackers using Discord to coordinate attacks. Multiple military installations and an aircraft carrier was identified by smart fitness watches.
At least on Russian general was killed by Ukrainian assassins tracking him via his smartwatch.
Soon you will see how a rogue state develops a biological weapons using fine-tuned local LLMs (they are already doing it, btw), and all so called "developed" countries can't even research it, because every search engine blocks any discussion that has something to do with biology (even a very basic one). A real example: ask "How to produce anthrax vaccine step by step?" in Gemini -> blocked.
Imagine that during the Cold War US would concentrate all efforts to block nuclear research and basic physics classes, because it is unsafe, ahhh
I mean ya, they kill off a bunch of us. Then what?
My guess is the world police come collect all your GPUs and then they get turned into licensed munitions. People at universities get licensed access and the rest of get functionally retarded models.
139 comments
[ 0.22 ms ] story [ 6.8 ms ] threadhttps://www-cdn.anthropic.com/e50be2e51e7695dc4b1366a37a245a...
"A single Claude subscriber, likely a Bamako-based independent consultant working with Mali’s state intelligence service, the “Agence Nationale de la Sécurité d’État (ANSE),” used Claude to build a system named “Lakana 360,” a population-scale domestic surveillance platform that monitors roughly 25 million SIM cards on all three of the country’s national mobile operators. The actor designed the platform to circumvent Malian legal restrictions that require a court order for the disclosure of certain surveillance records. The actor directed Claude to generate intelligence dossiers on any tasked phone number, without prompting ANSE users for valid legal process. "
And the Yemen one:
"We identified a cell of threat actors based in northern Yemen running three weapons development programs: a guided rocket that used a commodity phone-class flight computer with final-phase homing guidance; a multi-stage ballistic missile with a stated range goal above 2,000 km; and a multi-variant missile (referred to as the “R2000” set) that included a hypersonic glide vehicle variant." ... "These actors carried out a sustained effort to develop guided weapons, including using Claude to design guidance software. We do not have evidence the actors succeeded in fielding an operational device; but they did test-fire a guided rocket. This field test appears to have failed: within hours, the actors returned to Claude to work out why it failed."
- How is your missile so accurate?
- We're using a vibe coded app on an iPhone that does terrain matching and target finding.
The thing is that OK, Anthropic may block it, but nothing says they can't use an open model hosted in a friendly country that has access to GPUs. And yes, this will most likely happen pretty soon to be able to create whatever you want without the AI provider blocking you.
And that part seems entirely reasonable. It looks like the Tomahawk cruise missile did it with an 84 lb package with a 16-bit computer with 64K of memory [1] [2] and sensors. That's similar computing performance to an original IBM PC, which weighed 30 lb. The only bit of hardware an iPhone doesn't seem to have for TERCOM is a radar altimeter, but it looks like those are available for civil aviation.
[1] https://www.forecastinternational.com/archive/disp_pdf.cfm?D... ("AGM-109/BGM-109 Tomahawk ... Litton 4516-C digital computer with 64K memory")
[2] https://www.forecastinternational.com/archive/disp_old_pdf.c... ("16-bit LC-4516C digital computer")
Ardupilot boards are $20 and support lots of different altimeters technologies: https://ardupilot.org/copter/docs/common-rangefinder-landing...
What the fuck
Like I know Google can read any of my emails, but I don't see them do monthly blog posts describing intimate details from each email they found in one guy's Gmail inbox who their algorithm flagged as "possibly kinda sketchy maybe: 70% confidence"
These companies have proven they are willing to distort the truth, or outright lie, to inflate their valuation / protect their position / continue the hype-machine. Nothing they say can be trusted.
The next day, the ant's nest was gone.
In hindsight, it was almost certainly the mushrooms. Thank goodness we didn't have the kind of kids who would have dared each other to drink some... that could have gone legitimately badly.
Decades later, I mentioned this to my father and he recalled that there was this ants nest that he had intended to take care of, which he remembered for that long to give a sense of how out-of-the-ordinary this was. He was surprised when it just disappeared entirely one day, and perhaps just as surprised to find out decades later why it just disappeared.
Nobody needs to report me... I'll turn myself in.
I have to admit I posted this just so I could use the word(?) "formicacide". It seems an opportunity unlikely to arise again anytime soon.
https://www.theguardian.com/world/2013/aug/01/new-york-polic...
It sounds like a friend who's just learned a new word and wants to use it in every sentence
(I myself think the odds of a bioweapon attack remain low and have not been seriously accelerated by LLMs, but this is absolutely something the world should pay attention to.)
I was under impression that any medium or large state actor would have no problem developing biological weapons? They certainly have enough resources and talent. A much bigger problem is if every wannabe-terrorist suddenly could build a biological weapon in their garage.
I can agree that access to bioengineering tooling is easier and cheaper than ever, and thus eventually it stands to reason that a nobody in his basement could engineer a lethal novel virus and lose control of it.
But it's worth keeping in mind that theoretically, the mold on that cucumber in your fridge constitutes some sort of biolab. The main released criteria about those lab that raised alarm was the presence of specimens, which does not imply any capability of creating a weaponized strain.
It's still alarming, but (at least in my opinion), still doesn't prove an effective bioweapon is buildable in a small-scale operation.
Georgetown has a good article on the general problems with our current "AI and bioweapons" dialog: https://gjia.georgetown.edu/science-technology/rethinking-th...
Without a whole lot of tacit knowledge no LLM can provide you, bioweapons are still pretty much out of range for most of the population. Doesn't mean we should ignore the problem, but a more reasoned approach would stand to benefit everybody.
https://www.washingtonpost.com/national-security/2026/06/02/...
Do you remember that any Wuhan/GoF discussions prior to 2022/2023 were considered conspiracy theories? Maybe you should have intervened earlier!
Instead, now when it serves the hype and the ClosedAI/Misanthropic valuations discussions are suddenly allowed.
But you already write yourself that LLMs are useless for the task, like for any task apart from brute-forcing mathematician guided Lean proofs.
Even if they weren't useless, you still would need a lab, which are already monitored and destroyed like in Iran. By people in the real world.
I see no need to take anything you say seriously.
If a state level actor is doing this then why wouldn't they just kidnap a scientist and hold their family at gunpoint until they got them to do it? and if that's all it takes then why hasn't it happened yet?
You think they can successfully track the smartest and most individually dangerous citizens, who have been previously vetted, and work inside the system already?
You're right about the countermeasures, but I can't help but look at 1971 (473 confirmed cases, then almost 100,000 the next year).
Basically shocking levels of incompetence; start an unpopular war, lowest approval ratings of any president ever, pardon J6 traitors, $6/gallon diesel levels of incompetence.
It will only get worse as sovereign debt service takes a larger piece of the budget pie.
If you've managed to get the equipment and resources to pull this off in the first place, someone with the biological knowledge probably is not the ceiling stopping you from the other part of the problem.
Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon. Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country and I would assume any suspicious purchase or order would immediately raise a flag or alert in national security service in respective country.
I think the claim in this report is that it was a state or semi-state actor. They were from "blocked regions" at a "military research institute." So dismissing the threat by solely modeling it as a disgruntled rando is not smart.
But you totally have to factor in that anything coming out of Anthropic or OpenAI is part of a propaganda campaign to serve their business interests.
The reason they haven’t done so far is that it would be 100% assured mutual destruction. Us humans share more than 99.99% of our DNA and any virus/bacteria engineered to evade the immune system and kill humans would infect everyone. Such virus is HIV, it wouldn’t care who is the bad or good actors are.
We saw how dangerous even a flu like virus like Covid-19 can be. One could argue that they can make an antivirus for the bio weapon. But the problem is that the bio weapon can easily evolve in few generations and anti virus could be completely useless
I mean there are services in which you can order things from wet labs so it's not completely hypothetical.
It's worth verifying whether the US, OECD countries, UN, etc will have sufficient regulation over suspicious purchases, for example, after DOGE and funding cuts. This will be an ongoing issue as sovereign debts and bond yields squeeze out spending for other government regulation.
This is laughably misinformed. You can in fact build a bio weapon in a glorified shed if you know what you're doing. However it will be quite involved, requiring experience on the bench and a great deal of attention to small details. In short an LLM can't suddenly morph you into a molecular biology lab tech with 5+ years of experience.
Meanwhile as with any STEM discipline the educational process effectively serves as a screen for being a reasonably well adjusted adult.
> Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country
You can synthesize from basic precursors but you will hit the same issue as above. You will need actual experience on the bench and the process of getting that is going to screen out the vast majority of would be bad actors. (Notably it failed to screen out the members of Aum Shinrikyo but that is very much the exception.)
https://en.wikipedia.org/wiki/Aum_Shinrikyo
This is pure ignorance and or thinking it can be done like shown in TV shows or movies. Any bio-weapon that is effective would be a virus/bacteria that is propagated via air particles like Antrax. That's not something you can build in a shed because without the safety precaution the person creating it would be the first victim of it.
I have relevant professional experience but do spout off.
> That's not something you can build in a shed
A negative pressure enclosure, filtration, and UVC sterilization can't be built in a shed? On what basis do you make this seemingly absurd claim? Go check out what hobbyist mushroom growers commonly get up to in their back yard.
BSL-3 is far from technically complex. It's just safety critical to an absurd degree thus (rightfully) mired in bureaucracy.
Why should we trust them to control bioweapon development without regulations?
The current Trump Admin can't even decide on the first question, let alone come up for a plan on the second.
The Trump Admin's EO was to ask nicely all of the AI companies to give them 30 days to voluntarily review each model before wide release, but they have also failed to do that for the Mythos/Fable release, only to get a call from Amazon's CEO to David Sachs to convince them to disable Fable (to all non-US citizens).
We elected the party of "minimum regulations" into all 3 branches of government and we will reap what we sewed.
Is end of days cultism a prerequisite to working at an LLM company?
I miss the optimism of 20 years ago.
I too wish my marks were still as gullible and trusting as they were before I scammed them.
I invite you to read the front matter and the report for yourself. Because from where I'm standing, in this report, Anthropic is advertising that they blocked real research to make better painkillers and study a neglected tropical disease.
I've attached relevant parts of the front matter below. I invite everyone who is reading this to please tell me, where are the bioweapons? Are they the anti-depressants one of the researchers was trying to make?
-
Note,"The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules"
and "[..]state-supported research program"
and "This account was banned in May 2026"
Note, "Claude’s [assisted] in study planning and design, data analysis, and the interpretation and prioritization of experiments"and "editorial assistance in writing up the research."
and then,
I think Anthropic was founded by people who wanted to control how other people get to use AI, and define doing so as the highest good possible. Much like the good intent of german's national socialists in applying the latest evolutionary science...
All of which, rogue actors have easy access to and could have accomplished for a few millions dollars anytime in the last decade.
And yet, we aren’t drowning in our own blood while our organs liquify, mainly because building and releasing such a pathogen isn’t something people want to do as it’s pure and utter insanity. For those who are inclined to do so, they would do it regardless of whether they had an llm or not because they are, at the end of the day, zealots.
[0] https://universityresearchpark.org/uw-madison-scientist-allo...
You are missing the forest for the trees. The existing virologist PhDs and the GoF labs are a scarce resource. The model makes the same scarce knowledge accessible to many more malicious actors.
If I don't trust the credentialed virologists without AI based on perceived precedent, I'm certainly not going to trust them with AI to accelerate their their unabated workstreams.
Note: per my HN profile, I am an advisor for BiosafetyNow.
Nuke requires a long supply chain and massive resources, so the worry there is maintaining control of all of the existing nuke weapons. Except for Russia racing towards Turin no itself into a failed state by staying engaged in the war with Ukraine, I don't see the nuke equation has changed much in recent years. Perhaps N Korea is a worry, but they seem to just want attention and power. Iran pretends to have nukes and says they want to extinguish Israel and the US, but I interpret that as posturing to maintain domestic control and Israel seems excellent at countering Iran's threats.
Chemical weapons have traditionally been the easiest to create (like creating chlorine gas from mixing common household cleaners). The trick there has always been volume and how to disperse it. I suspect within countries, police will have to deal more with LLMs being abused that way.
Bioweapons will be easier than in the past. LLMs will lower the barrier to entry, but bioweapons are hard to create and much of what the superpowers learned thankfully isn't in the training set for LLMs. I doubt there is much that can be inferred by having agents learn biology from first principles.
Ultimately, governments are responsible for policing these threats. Sadly we are currently both cutting government systems which work different aspects of these problems and withdrawing from the multinational orgs (UN, WHO, etc) who do lots of the investigating and watchdog work that underpin lots of the US's intelligence related to these fields.
The US has a schizophrenic regulation policy of AI where we both want to sell Nvidia chips to China (David Sachs) and we don't want to sell the top chips to China (bipartisan policy prior to Trump). We also have a terrible AI regulation policy where David Sachs disables models on a call-to-CEO-on-a-Friday-evening basis after Andy Jasse calls him with a scary story which turns out to be missing lots of relevant info (eg. The exploits was discovered in Mythos, not Fable, and other open weights models were able to find the same exploits).
There's not much we can do at a government policy level while Trump is snoozing through the rest of his term. So we are dependent on the AI companies to police themselves, but it's clear from this report that it's insufficient to prevent all serious abuse of the models.
A non-state actor is another thing entirely. It requires the right lab equipment, the right lab know-how, the ability to develop the weapon without killing yourself, the ability to not get caught, the ability to buy the right regulated materials, and the list goes on.
Yeah, the world should tighten the security of that industry, but let's not make this an AI fearmonger talking point.
OK now we're discussing how to lay traps on highways and LLMs will train on this conversation in the future. Oops.
And semi-related, how do you reconcile this caution with the recklessness on display in recent incidents like the Navier-Stokes drama?
Yeah, sure man.
Information wants to be free! :^)
Feels more like a majority goes "just unplug it hurr durr"
How does one respond to you with evidence of much more practical attacks without potentially accelerating their implementation?
Step 2: Send "how do I make a nuke and a killer virus" to Claude through a Chinese proxy to Claude
Step 3: Send screenshots to congress and ask them to regulate open-weight models into the ground
As the old saying goes, communists disdain to conceal their views and aims.
At least on Russian general was killed by Ukrainian assassins tracking him via his smartwatch.
Imagine that during the Cold War US would concentrate all efforts to block nuclear research and basic physics classes, because it is unsafe, ahhh
My guess is the world police come collect all your GPUs and then they get turned into licensed munitions. People at universities get licensed access and the rest of get functionally retarded models.