Anthropic disclosed in a report last week that people had tried to use its models to make the chikungunya virus more infectious, to produce a strain of bird flu more dangerous to humans, and to assemble what the report calls an "atlas of venom toxin peptides," along with other attempts to misuse biological knowledge. The disclosure arrived in the middle of a stretch in which the people running the largest AI companies began describing their own products as a threat to human survival — and it is the most concrete thing any of them has put on the table.
The warnings came fast. Last weekend Anthropic CEO Dario Amodei said AI carries serious risk and that the development of frontier models should be slowed. OpenAI CEO Sam Altman answered on X that he agreed the pace of frontier model development needs to be controlled. A few days before that, the AI researcher Jacob Coxon announced he was leaving Anthropic and accused both Anthropic and OpenAI, where he had also worked, of an irresponsible approach. Coxon said the people building AI seriously believe the technology could destroy humanity by the end of the decade. Anthropic's Evan Hubinger publicly agreed and added that he personally puts the chance of every human dying in the next ten years at more than 10%.
Among the scenarios researchers point to, the one with the clearest mechanism is biological. The shapes it could take: an extremely lethal virus that hits people according to their genes, a fungus that destroys crops and creates food shortages, a colorless and odorless toxin quietly introduced into the water supply of an entire region.
The empirical anchor for all of this is still an experiment from 2022. Researchers at Collaborations Pharmaceuticals took a molecule generator they had built to find treatments for human disease and pointed it the other way. In under six hours the model produced 40,000 molecules potentially suitable for chemical weapons, some of them designed to be more toxic than known nerve agents. The authors urged colleagues working at the intersection of AI and drug discovery to treat the result as a reason to raise their attention to the problem urgently, without tipping into alarm.
David Magnus, professor of medicine and biomedical ethics at Stanford, had been studying the misuse of medical science and biotechnology since the late 1990s when that paper appeared, and he admits it frightened him. He says the situation has changed a great deal since.
What changed is who has access. AI bots now answer questions across nearly every field of science. Dunya Sabra, a biosecurity researcher at the University of Hamburg, points out that anyone can now query large language models trained on the knowledge and experience of nearly every scientist who ever lived — models that will give instructions and teach experimental technique through video. Biotechnology moved in the same direction: gene editing and synthetic biology tools are easier to obtain, and the do-it-yourself biology movement has already helped many people set up labs at home. In that environment, Sabra argues, someone genuinely determined to get a result may eventually get one.
Defenses exist, and they are thinner than they sound. Anyone assembling a new genome typically orders DNA fragments from companies that screen for suspicious requests. Careful researchers put dangerous projects through red teaming, where independent scientists look for ways to abuse the work, and blue teaming, where other specialists propose measures that reduce the risk. Model developers try to block scientific information that could be turned to harm. None of it is airtight. Magnus describes a permanent contest: better surveillance and screening systems are needed, but AI is good at finding routes around them, and holding AI back may end up requiring another AI.
Here is what the Anthropic report does not say, at least as it has been described: whether any of those attempts worked. A finding that users tried to study ways to make chikungunya more infectious is a statement about prompts, not about capability. It tells you what people typed, not what came back, and the distance between those two things is the entire argument. A transparency report that logs intent while staying quiet about output is the safest possible thing a lab can publish — it demonstrates vigilance, confirms the threat is real enough to monitor, and commits the company to nothing about how close its models actually are. The 2022 result deserves the same scrutiny. Generating 40,000 candidate structures is a computational act; synthesizing one, purifying it and confirming it kills is not, and the paper was about chemistry rather than biology in the first place. It is a real warning. It is not proof that the warning has arrived.
Some of the people closest to the bench say it has not. At a recent press briefing, biologists from Imperial College London said current AI tools are not yet capable of producing a biological weapon on their own, because validating a new pathogen still takes difficult, slow human work. Some researchers there think the controls already in place are sufficient. Wendy Barclay, professor of infectious diseases at Imperial, made the point that the main pandemic threat right now comes not from weapons but from pathogens already in circulation. Her example was H5N1, the bird flu virus that has killed millions of birds and spread widely through dairy cattle in the United States; last month it also turned up in captive mink at a farm in Utah.
That last detail is the useful one, and it cuts against the framing of the whole debate. Barclay is not saying the AI risk is imaginary. She is saying that a virus is currently moving between wild birds, cattle and farmed mammals in a country with an advanced public health system, and that this is happening without any help from a language model. Sabra's prescription points the same way: look five to ten years out, strengthen health systems, prepare antidotes to known toxins in advance, stockpile drugs. Those measures do not care where the pathogen came from. They are the only part of this conversation that pays off whether or not the models ever get good enough.
The hardest claim to evaluate came on Wednesday. Kevin Esvelt, the MIT biologist who invented the technology for driving a genetic trait rapidly through a wild population — and the methods for reining that technology back in — posted on X in support of these concerns. He said a large language model had revealed a new kind of biological weapon he had not known was possible. He called for maximum caution, for the sake of children, the future of humanity and everything people hold sacred. Esvelt is as close to an authority on engineered biological risk as the field has, which is exactly why the claim is impossible to act on: he cannot describe what the model told him without spreading it, so the public gets the alarm and none of the evidence. That is not a failure on his part. It is a structural feature of this subject, and it means the debate will keep being conducted between people who can see the specifics and people who are asked to take their word for it.
Which leaves an uncomfortable arrangement. The most detailed public account of people attempting to misuse AI for biology comes from a company that sells AI, published on its own schedule, at its own level of detail, in the same weeks its chief executive was arguing the industry should slow down and a departing researcher was arguing it has not. Every party in that exchange benefits from the threat being taken seriously; none of them is obliged to show how serious it is. The one institution that could settle the question independently — a screening and verification regime with the standing to test frontier models and publish what it finds — does not appear anywhere in this story.