i
News
News · 2026-10-19

Anthropic’s AI shutdown proposal sets a 50-person threshold

@neuronium_ai @neuronium_ai

Anthropic is privately discussing a bill that would let the federal government halt an AI model only when it poses an imminent threat: a risk of killing or seriously injuring at least 50 people, or causing more than $1 billion in property damage. The proposal, reported by Politico, would limit intervention to three scenarios and put the final decision in the hands of a federal judge. That narrow threshold matters: it offers a way to stop a model while leaving labs considerable room to operate.

Cover: Anthropic’s AI shutdown proposal sets a 50-person threshold

A high bar for intervention

The proposal would apply only if a model could lead to one of three threats:

The release of a biological weapon
A catastrophic cyberattack
An AI model escaping its lab’s control

An independent monitor overseen by the Department of Commerce would assess the risk. If the monitor found that a future model could cause one of those threats, it would have 72 hours to alert department officials. They would then decide whether to present the evidence to a federal judge, who would make the final call on whether the model could be released.

Politico reports that three people familiar with the talks described Anthropic’s role in shaping the bill. The company told the outlet the proposal reflects its document, “Frontier AI Framework,” and said it has advocated for the measures at the federal and state levels since June. Anthropic considers them reasonable and believes Congress should pass the bill.

The negotiations come as AI companies spend heavily on politics ahead of US midterm elections. Known AI companies have spent at least $55.7 million, while executives at leading labs, including OpenAI and Anthropic, have directed tens of millions of dollars to political groups. Their specific positions on regulation had remained unclear.

What the threshold leaves out

I think the most consequential feature is not the power to halt a model, but how tightly the proposal defines when that power can be used. The 50-person and $1 billion thresholds, combined with three specified threats and a judge’s decision, make this a constrained intervention mechanism—not broad government authority over AI releases.

The announcement is quiet about how an independent monitor would assess the risks, or what evidence would be enough to persuade a judge. Those details could determine whether the process is a meaningful safeguard or a high bar that rarely changes what labs can ship. Anthropic’s proposal offers a clear boundary for intervention; its practical force depends on how that boundary is applied.

Daily AI news

Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.

Only what matters — every day

Follow on X