Anthropic Calls on Governments to Block Dangerous AI Models

Anthropic released two policy frameworks Wednesday that lay out what it wants from governments on AI safety and economic adaptation. The AI startup is asking for legal authority to block advanced models that pose catastrophic risks, along with a slate of measures to make society more resilient and to prepare workers for potential job displacement.

The company's Advanced Policy Framework calls for government power to halt AI deployments in four categories of catastrophic risk: biological threats, cyber attacks, loss of human control over the system, and automated research and development. That blocking authority, Anthropic said, should apply only to models above a certain capability threshold and only to companies above a specified size. The framework also argues that the federal government should not preempt state law unless it passes a sufficiently strong federal law itself.

Governments need more than transparency requirements when dealing with these risks, Anthropic said. They should also invest in early-warning biosurveillance to spot novel outbreaks, in widely shared cyber capabilities, and in the ability to detect and shut down AI systems that act outside their developers' control.

The Economic Policy Framework addresses three unemployment scenarios. At 5% unemployment, Anthropic proposes workforce training grants, occupational licensing reform, wage insurance, and incentives for firms that retain or redeploy workers. At 10%, it calls for expanded unemployment insurance. For what it calls “unprecedented levels of unemployment,” the company argues for sustaining income replacement for a large share of the workforce.

Anthropic has been pushing for restraint on AI development speed. Last week the company suggested that frontier AI developers should slow or pause their efforts to give societal structures and alignment research time to catch up. That proposal was triggered by the growing share of AI development being delegated to AI systems, and the possibility that an AI system could fully autonomously design and develop another AI system. Such a development, Anthropic said, “might increase the risks of humans losing control over AI systems.”

The policy documents represent the company's most detailed attempt to shape the regulatory conversation, coming as governments worldwide weigh rules for advanced AI. Whether lawmakers will adopt the specific triggers on model capability and company size remains an open question.