Anthropic AI policy framework seeks legal powers to curb high-risk AI launches

Anthropic has proposed a new AI policy framework under its “AI Exponential” plan, arguing that frontier model progress is outpacing current regulation. The AI policy calls for governments to have authority to block or deter dangerous AI deployments through civil penalties tied to global annual revenue, with higher penalties for repeat violations. The proposal splits into two tracks: an Advanced AI Framework for frontier model safety and an Economic Policy Framework focused on workers and shared financial benefits. It would apply only to the most capable systems—set at models trained above 10^25 floating-point operations—and to companies earning over $500 million in AI-related revenue (or spending more than $1 billion on AI R&D). Safety requirements include mandatory pre-release testing, publication of safety documentation (summaries, safety frameworks, system cards), and independent evaluators who review risk reports. Developers would also need stronger security for model weights and training systems, plus public reporting on overall risk posture. Risk coverage targets catastrophic scenarios: biological risk (e.g., misuse for harmful virus development), cyber risk (large-scale vulnerability discovery affecting hospitals and power grids), loss of control, and risks from automated AI research. A second component emphasizes public resilience: gene synthesis screening and biosurveillance for biological threats, stronger internet software and critical-infrastructure support for cyber threats, and improved tools to detect, contain, or shut down unsafe systems. Anthropic says policymakers should start with lighter rules and adjust over time as capabilities evolve.