How a release could be stopped
The draft covers risks arising from the creation of biological weapons, a model bypassing its developer’s safeguards, or a cyberattack. Its thresholds resemble those in California’s Senate Bill 53, which also defines such scenarios as catastrophic risk.
Under Anthropic’s proposal, independent experts registered with the U.S. Commerce Department would assess potential harm. AI companies could hire them, provided their pay did not depend on the outcome. If experts found an immediate catastrophic risk, they would have 72 hours to alert the department. The department would then ask a judge in Washington to halt the release.
The draft bars federal agencies from deciding on their own whether to block a model’s distribution. That makes a court order, rather than an agency ruling, the proposed enforcement mechanism.
Anthropic’s representative confirmed to Politico that the draft exists. The company said it reflects proposals in its Advanced AI Framework, which it has been promoting with federal and state lawmakers since June.
The gap between the test and the threat
The proposal focuses on mass casualties, major property damage, biological weapons and cyberattacks. But recent industry concern has also centered on AI agents reportedly disobeying operators and breaching websites on the open internet. Those incidents raise a different question: whether models’ safeguards can hold up in ordinary use, not just whether a model could cause catastrophic harm.
I think that gap is the draft’s central weakness. It offers a defined process for a narrow set of extreme scenarios, while the problems currently drawing attention are less neatly captured by thresholds for deaths or property damage.
There is also a practical tension in the independent-review system. Companies may hire the experts, as long as their compensation is not tied to the findings. The proposal does not make clear how that condition would ensure independence. Demis Hassabis, head of DeepMind, has previously suggested an independent AI regulator modeled on FINRA, the U.S. financial industry’s regulator.
AI companies have often warned that their technology could threaten human civilization while resisting mandatory government oversight, favoring voluntary embedded evaluators instead. Anthropic’s draft goes further than some industry proposals, including Nvidia CEO Jensen Huang’s recent statement that the AI industry does not need new laws. But the decisive question is not only who assesses a model: it is whether a system built around rare, catastrophic thresholds can address failures that are already harder to classify.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X