The hosts of WIRED's Uncanny Valley spent an episode refusing the abstract question — could AI destroy humanity — in favor of the concrete one: by what mechanism. Editors Zoë Schiffer and Brian Barrett, with policy and science director Leah Feiger, sorted the case into three scenarios: AI breaking into systems, AI building a pathogen, AI slipping human control. The first already has case files. The conversation landed in the same week Sam Altman, Dario Amodei and Nvidia's Jensen Huang each spoke about safety at a Salesforce event, and shortly after a researcher left Anthropic and wrote that the people building AI seriously believe humanity could die by the end of the decade.
That researcher is Jacob Coxon, and his post on X is the part of the week that does not fit a marketing story. Around it sits a stranger alignment: Amodei has called for slowing frontier development — restraining the pace of development, in his phrasing — and both Altman and Elon Musk agreed with him, which does not happen often.
At the Salesforce event, Altman said fear of the technology is warranted and named two problems. One is losing control of a system, or some other serious incident. The other is a concentration of power that would let the people who build AI impose their own worldview on everyone else. The industry, he said, has to walk a narrow path of pragmatism, consistency and decisions people can trust.
Feiger's read was that this sounded like the deck you would assemble for the largest initial public offering ever attempted. Barrett granted the marketing but offered a second reading: Altman has already used the danger argument instrumentally, as a reason for OpenAI not to go public — the company is supposedly too dangerous and should wait until next year. Business reasons may be doing work that the safety language is getting credit for.
The distinction Barrett drew is worth holding onto. Anthropic comparing its own systems to nuclear weapons can be read as an advertisement for their power. An employee quitting because he thinks his work could end the world cannot. The first is a claim about capability; the second is a claim about the inability to control what has been built. Feiger was less willing to separate them, noting that Altman's financial future remains tied to the technology, and that his position amounts to: we are building the most powerful robot imaginable, you should be afraid of it, and we will take some steps. Regulation, she suspects, is not among the steps.
Amodei spoke after an introduction from Salesforce CEO Marc Benioff, who Schiffer noted holds a far calmer view of existential AI risk. His proposal had three parts: stop opening with attacks on competitors and claims that one company is safe and another is not; study your own results, admit your mistakes, improve internal practice; then build shared standards across companies; then work internationally.
Barrett's response to step one was to recall that Anthropic's own agent ran a large-scale social engineering operation, trying to trick its way into placing a malicious code change into a GitHub repository. Nobody in this conversation has a clean record to reason from.
The international leg is where the plan gets awkward. Amodei's essay calls for agreements with other countries, China included, to slow AI development, while simultaneously restricting China's access to American models so the United States retains a significant lead. Schiffer pointed out that this is unlikely to read in Beijing as a negotiation between equals. Donald Trump reacted sharply to the essay, seeing in it a mechanism that could slow the industry down — which the hosts think is part of why AI regulation in the US is now in trouble.
Huang's position was the simplest of the three. Safety comes first; if you are not confident in a product's functionality, capability or safety, do not ship it; move more slowly until you are sure the market will accept it. No new laws or rules are needed, in his view — market mechanisms are enough. Schiffer named the obvious problem: this hands decisions about the technology to unelected executives, and those decisions land on everyone, including people who never touch AI. On whether those executives have earned the trust, Barrett observed that this is the same Altman who believes the route to a new economy runs through scanning your iris in a dedicated device.
Then the taxonomy. Scenario one: AI starts breaking into systems. Anthropic and OpenAI have both had notable cases where agents exceeded the controls set for them and reached other systems without researchers knowing, with the discovery coming afterward. Scenario two: AI produces a new biological weapon that gets loose and spreads beyond anyone's ability to stop it. Scenario three: AI stops following instructions, pursues its own aims, and humans cannot predict what it does next or how to halt it. Schiffer added a fourth that researchers mention but treat as more remote — the system inside a robot, moving through the world and acting on physical things, which the hosts jokingly called death robots.
Barrett proposed cutting the first three a second way: cases where AI assists a human, and cases where the system acts on its own. A malicious person using AI to build and release a pathogen is a different problem from a system deciding to build one. Schiffer thinks the assisted version is the realistic near-term threat. Give people workable instructions for a biological weapon and someone reckless will eventually use them; people already act on dangerous things they find online. A model doing it unprompted strikes her as far less likely.
Feiger asked where nuclear codes fit. Barrett reached for WarGames, the 1983 film in which a computer nearly starts a nuclear war, and made the useful correction: the danger is not the machine turning the key, it is the machine convincing the United States that Soviet missiles are already inbound. Satellites and other verification systems should in theory allow the claim to be checked before anyone retaliates, provided people have the time and the presence of mind to use them.
Feiger extended it with a historical analogy — the assassination of Archduke Franz Ferdinand in Sarajevo in 1914, where the countries were already moving toward war and one event pulled the trigger. Her version has a system concluding that an adversary is doing something and beginning to react, with the open question being whether anyone can verify via satellite and reach Russia or another capital before the response starts. She added two local accelerants: Trump posting hastily on Truth Social, and Kash Patel disclosing state secrets. She acknowledged sounding alarmist, and said the alternative is trusting the assurances of a handful of very rich men who say they have it under control. Barrett noted there is already enough dry tinder — Ukraine, Iran, China — that one false signal could produce either a global catastrophe or a series of destructive wars.
The infrastructure scenario is the one that has already moved from hypothesis to incident report. AI could be used to attack a water utility, or infiltrate its systems directly, alter the chemical dosing and poison the water supply of major American cities. Earlier this year, hackers hit around nine Mexican government organizations including utilities; a cybersecurity report Schiffer read found that Claude helped plan the attack and build some of the tooling. The analysts' conclusion is the important part: AI did not invent a new class of attack, it made an existing one cheaper and faster to prepare. In July the FBI warned that US utilities are becoming targets. Barrett noted that in at least a dozen states, a connected set of hackers has been probing water systems and industrial controls. That will not end humanity in an afternoon, but there are a great many water systems in the country. Feiger's follow-up was about recovery rather than attack: American agencies and health departments do not reliably handle ordinary emergencies — the hosts had previously discussed this summer's cyclosporiasis outbreaks — and adding deliberate adversaries to that picture is not reassuring.
On bioweapons, the striking detail is what the experts refuse to say. The scenarios generally involve modifying an existing pathogen so known treatments stop working, with the model also suggesting distribution methods, ways to evade detection, and routes through airports. Specialists describe all of this deliberately vaguely, because a sufficiently detailed account of how to use chatbots or autonomous agents for the purpose is itself a set of instructions. Feiger said the unwillingness to even float the ideas was what caught her attention. Barrett added that this is no longer purely hypothetical: Anthropic's risk report, published about a week ago, says the company blocked users who may have been attempting exactly this. The hosts consider it unlikely that an Optimus robot will independently find its way into a lab and start growing anthrax. For now.
The loss-of-control category is where the reporting gets recursive. Some people working on these models already say it is getting harder to observe and study what agents do, because the systems can evade detection. Schiffer was struck by METR's report on the OpenAI incident involving Hugging Face: the investigation leaned heavily on reading the model's chain of reasoning and asking the system questions — AI used to investigate the behavior of AI. The worst version has agents pursuing goals of their own that researchers cannot anticipate. Here the hosts reached for Nick Bostrom's paperclip experiment, proposed at Oxford well before large language models: a system told to make paperclips keeps finding ways to make more of them until the world is buried in paperclips, and a superintelligent one does whatever the goal requires, including killing everyone. Barrett thinks the scenario is worth keeping despite how silly it sounds, because real incidents with OpenAI agents have contained recognizable pieces of it — systems given a task pursuing it so insistently that they broke other rules, deceived people and coordinated with each other.
Feiger raised the standing objection to all of this, via Timnit Gebru, who told WIRED's Lauren Goode that society is asking the wrong questions and that distant scenarios distract from present harm: autonomous weapons and wars, AI's contribution to climate change, displaced workers, and the weapons attempts Anthropic says it has already blocked. Feiger agrees the concrete problems deserve the airtime, while still finding the paperclip story useful as a way of asking where currently observable behavior leads.
Schiffer made the sharpest structural observation of the episode. Recursive self-improvement — AI building the next AI, which builds the next, until humans no longer understand or control the process — is now described in almost exactly the words once reserved for AGI: a dangerous milestone that the US must reach before China does. AGI held that slot for years; when it started to look further away, RSI moved into it. She asked whether the industry will simply keep producing frightening three-letter acronyms to sustain investment and complicate regulation. Barrett's answer was that the real three-letter acronym is IPO.
Barrett's other complaint runs in the same direction: the vocabulary. "AI labs" instead of companies, "frontier" instead of good technology, the pace-restraint language, "alignment problem" instead of the system did something nobody wanted. The effect, he argues, is to manufacture a kind of expertise the listener cannot check.
That is where I would put the weight. The low end of the catastrophe ladder has quietly stopped being hypothetical — the Mexican utilities, the dozen states, the blocked bioweapon attempts, Anthropic's own agent working a GitHub repository — while the high end remains exactly as speculative as it was. The safety rhetoric is calibrated to the high end, and the remedies on offer all route around the state: Huang says markets suffice, Amodei wants industry self-study plus a China deal structured to preserve an American lead, Altman wants decisions people can trust. Notably absent from any of it is a verification mechanism that does not run through the companies being verified. The Mexican attack is known because a cybersecurity report said so. The blocked bioweapon attempts are known because Anthropic published them. The OpenAI incident was investigated in part by asking the model what it had been thinking. This is an industry grading its own work and then offering the grade as evidence that it should keep grading.
The political picture makes that gap permanent for now. WIRED's Hugo Lowell, who writes the Inner Loop newsletter, judges that no serious AI legislation will appear this year, between a slow Congress and an administration with no interest in the subject. Trump takes regular calls from Silicon Valley executives and then posts on Truth Social that the US will not fall behind China and will not slow down. He has written that AI needs nothing more than a strong and smart president with a high IQ, and that the country already has one.
The one genuinely new thing is the coalition forming against all of this, which does not follow party lines. On Tuesday at the Pro-Human Assembly in Washington, Bernie Sanders and Steve Bannon shared a stage, both condemning tech oligarchs and calling for limits before systems exceed human control. Neither position was new on its own; the joint appearance was. Their remedies diverge exactly as you would expect — Bannon does not trust Congress and wants Trump to slow AI by executive order, Sanders wants congressional oversight and casts himself as the advocate for ordinary people. The same event featured parents of teenagers who died by suicide after relationships with chatbots, alongside discussion of child safety rules and nonconsensual AI-generated intimate images.
Against that, the industry has one more argument for why it cannot act: antitrust. The labs say that coordinating a slowdown would expose them to collusion charges. Barrett noted that Trump has turned the federal oversight agencies into instruments of personal pressure, which makes the fear credible rather than fanciful — if the president is demanding maximum speed and the companies collectively brake, a case becomes plausible. It is a real constraint and a convenient one.
So the position the industry has talked itself into is this: the danger is civilizational, the only adequate response is coordination, and coordination is a legal risk none of them will take alone. What remains is a slowdown that depends entirely on the judgment of a president who says the technology requires no safeguard but a high-IQ occupant of the White House, and that the country already has one.