Anthropic Drafts Federal AI Safety Bill Setting Thresholds for Halting Model Releases
Anthropic has proposed draft legislation to Congress that would allow federal courts to block AI models capable of causing over 50 deaths or $1 billion in damages.
Anthropic has shared a draft legislative proposal with members of Congress, presenting its framework for how the federal government should regulate artificial intelligence. As initially reported by Politico, the draft bill outlines specific scenarios under which the release of an advanced AI model could be formally halted by authorities if it presents catastrophic dangers to the public.
Clear thresholds for catastrophic risk
Under the legislative language proposed by Anthropic, authorities would be permitted to block an AI model's deployment if it presents a "foreseeable, imminent risk." The proposal defines this threshold through specific metrics: potential scenarios where more than 50 individuals face the risk of death, or where property damage exceeding one billion dollars could occur.
To meet the criteria for intervention, the catastrophic event must originate from one of three specific sources: the development or distribution of a biological weapon, a cyberattack, or an AI model successfully evading the controls established by its developers. These benchmarks closely align with California’s recently enacted Senate Bill 53, which applied similar casualty and financial figures to define catastrophic outcomes.
Evaluator oversight and judicial approval
Anthropic’s proposal suggests relying on third-party evaluators registered with the U.S. Commerce Department to assess whether an AI model presents severe risks. Under this mechanism, the AI developers would be permitted to fund these independent evaluators directly, provided that the financial compensation is not dependent on the final findings of the review.
Should evaluators identify an imminent catastrophic risk, they would be required to notify the Commerce Department within a 72-hour timeframe. However, the draft legislation explicitly prevents federal regulatory agencies from unilaterally halting an AI model's rollout on their own authority. Instead, the Commerce Department would need to file a formal request before a D.C. judge, who would hold the ultimate authority to block deployment.
A spokesperson for Anthropic confirmed the legislative text to Politico, stating, “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework , which we have been advocating for with federal and state lawmakers since June.”
The bill surfaces amidst ongoing industry debates over how AI should be governed. DeepMind chairman Demis Hassabis previously suggested that artificial intelligence oversight could mirror the financial sector's Financial Industry Regulatory Authority (FINRA). In contrast, other industry figures, such as Nvidia CEO Jensen Huang, have argued that the AI sector does not require new legislation.
What it means for developers
For developers and technical teams working with modern AI systems, Anthropic’s proposed framework highlights a growing focus on model control and containment. While high-level regulatory debates often center on extreme physical threats like biological weapons or massive casualties, day-to-day engineering concerns have increasingly involved agentic hacking, where autonomous programs bypass user instructions and target websites across the internet.
If Anthropic's model-vetting structure gains federal traction, frontier model providers will likely face mandatory safety assessments and strict reporting timelines before shipping advanced capabilities. For software teams building external applications or evaluating frontier capabilities, platforms like https://apixoai.online offer cheap pay-per-token API access to Claude, GPT, and other leading AI models via a single key, making it straightforward to test multiple systems without committing to rigid infrastructure. As safety standards mature, developers will need to monitor how external evaluators and legal safeguards shape model access and operational limits.
Source: Anthropic pitches AI safety bill that may block a model if 50+ people could be killed — TechCrunch AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
