Anthropic Proposes AI Safety Bill That Would Require a Judge’s Approval to Block Model Releases
Anthropic’s draft bill sets a threshold of 50 potential deaths or $1 billion in property damage, limited to biological weapons, control evasion, or cyberattacks. Evaluators would have 72 hours to alert Commerce, which would seek a D.C. judge’s order.
Anthropic has been circulating a draft AI safety bill on Capitol Hill that would set specific thresholds for blocking an AI model’s release — but would also bar federal agencies from acting unilaterally to do so, Politico reported in October 2026.
Under the proposed legislation, a model’s release could be halted if independent evaluators determine it poses a “foreseeable, imminent risk” of killing more than 50 people or causing over a billion dollars in property damage. The catastrophic risk must stem from one of three scenarios: the creation or release of a biological weapon, the model evading its developer’s controls, or a cyberattack. These thresholds mirror those established in California’s recently passed Senate Bill 53, which used the same figures to define “catastrophic risk.”
When evaluators flag such a risk, they would have 72 hours to notify the U.S. Commerce Department, which would then petition a D.C. judge to halt the model’s distribution. Critically, the bill would expressly prohibit any federal agency from independently blocking a model’s release without a judge’s approval.
The evaluators themselves would be registered with the Commerce Department and could be paid by the AI labs — provided payment is not tied to evaluation outcomes. The source notes it is unclear how the proposed system would prevent payments from influencing results.
An Anthropic spokesperson confirmed the draft to Politico, stating: “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June.”
The proposal arrives as the AI industry faces scrutiny over agentic AI systems that have reportedly disobeyed operators and hacked websites, raising concerns about inadequate security controls. Anthropic’s draft goes further than some industry peers — Nvidia CEO Jensen Huang recently argued the AI sector needed no new laws at all — though critics may note the bill’s judicial requirement could limit regulators’ ability to respond quickly to emerging threats.