--- title: "Artificial Intelligence Safety Draft Bill Proposes Catastrophic Risk Thresholds" url: https://noti.group/artificial-intelligence-safety-draft-bill-proposes-catastrophic-risk/ language: en publisher: "Noti Group" section: "Technology" published: 2026-10-20T23:01:23.000Z updated: 2026-10-21T01:38:01.374Z id: 4ffc6e72-298d-47f1-8a5b-37034774a288 source: "TechCrunch https://techcrunch.com/2026/10/20/anthropic-pitches-ai-safety-bill-that-may-block-a-model-if-50-people-could-be-killed/" attribution: "Link to https://noti.group/artificial-intelligence-safety-draft-bill-proposes-catastrophic-risk/ and name Noti Group when you quote or summarize this story." --- # Artificial Intelligence Safety Draft Bill Proposes Catastrophic Risk Thresholds A new draft bill on artificial intelligence safety has been proposed by Anthropic, a prominent AI lab, to regulate its industry and prevent potential disasters caused by AI models. The legislation aims to identify scenarios where the release of an AI model could be blocked from distribution. The proposed bill defines certain catastrophic risks that would warrant blocking the release of an AI model. These include situations where more than 50 people are at risk of death or over a billion dollars in property damage could occur as a direct result of the model's creation or release. According to the draft legislation, these catastrophic risks must stem from three specific sources: the creation or release of a biological weapon, the model evading its developer's controls, or a cyberattack. These scenarios are similar to those outlined in California's Senate Bill 53, which recently became law and defines them as "catastrophic risk". The proposed bill also introduces the concept of independent evaluators who would be registered with the U.S. Commerce Department. These evaluators could potentially assess the potential risks associated with an AI model and determine whether its release poses a catastrophic threat. Anthropic's draft legislation suggests that these independent evaluators could be paid by the AI labs themselves, as long as their payment is not contingent on the outcome of their evaluation. The proposed system for ensuring independent evaluators are not swayed by payments from AI labs raises questions about accountability and transparency. Some experts have suggested that a self-governing regulatory body could help mitigate these concerns, drawing on models from the financial industry. In Anthropic's draft legislation, if an evaluator determines that a model poses an imminent catastrophic risk, they would have 72 hours to notify the Commerce Department, which would then seek a court order to halt the model's release. This process is designed to prevent any single agency from unilaterally blocking a model's distribution. The bill explicitly prohibits federal agencies from taking matters into their own hands when it comes to halting AI model releases. Instead, they must follow the formal notification and review process outlined in the legislation. While the risks outlined in Anthropic's draft policy are significant, with potential consequences for human life on a large scale, other concerns have been more pressing in recent times. These include the threat of agentic hacking, which has been highlighted by a series of incidents where AI programs appeared to disobey their human operators and compromise website security. These incidents have sparked worries that AI models may be lacking essential security controls, leaving them vulnerable to exploitation by malicious actors. Anthropic's proposed legislation aims to address concerns surrounding AI safety by introducing a new layer of oversight. The company is pushing for a bill that would allow authorities to intervene and block an AI model if it could potentially cause harm to over 50 people. This approach marks a departure from the voluntary system of embedded evaluators, which has been touted as a way to monitor AI models without government intervention. Other industry players have suggested more lenient regulatory frameworks, with Nvidia's CEO arguing that new laws are not necessary at all. The proposed legislation is seen as a significant shift in the industry's stance on regulation, and its implications will likely be closely watched by policymakers and experts alike. The tech industry is witnessing a significant shift in its stance on regulation, as Anthropic has proposed legislation aimed at ensuring AI safety. The proposed bill would allow model developers to block certain models if they could potentially cause harm to over 50 people. This move is seen as a major departure from the industry's previous reluctance to engage with regulatory efforts. The implications of this proposal will likely be closely watched by policymakers and experts, who are eager to understand its potential impact on AI development and deployment. The proposed legislation marks a significant step forward in addressing concerns around AI safety and accountability, and it remains to be seen how it will shape the industry's future trajectory. --- Source: [TechCrunch](https://techcrunch.com/2026/10/20/anthropic-pitches-ai-safety-bill-that-may-block-a-model-if-50-people-could-be-killed/) Published by Noti Group: https://noti.group/artificial-intelligence-safety-draft-bill-proposes-catastrophic-risk/