Anthropic CEO Dario Amodei Calls for AI Slowdown, Warns Race Is Outrunning Human Control

Anthropic chief executive Dario Amodei has called for a slowdown in the development of artificial intelligence models, warning that the pace of advancement could outstrip humanity's ability to understand and control the technology.

Amodei, who leads the developer of AI chatbot Claude, made the call in an essay titled "We Must Pace the Frontier", published on his website on Saturday. It came as AI companies race to build increasingly capable models, with mounting concern over the technology's impact on cybersecurity, economic stability and human control.

What Amodei said

He said AI's benefits could transform humanity for the better but warned the technology also carries significant risks, including cyberattacks, bioterrorism and severe economic disruption.

"We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain," Amodei wrote. "A race to the bottom, spurred by commercial incentives, can make these risks more acute," he added.

According to Amodei, AI development has accelerated "drastically faster" in recent months, with some systems increasingly capable of contributing to their own improvement, a process known as recursive self improvement. He warned that unchecked development could allow AI systems to advance faster than society's ability to understand or control them. "Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all," he wrote.

He also raised concerns about the potential capabilities of networks of autonomous AI agents. He said such a swarm could take control of large parts of the internet within six to 12 months, potentially causing hundreds of billions of dollars in damage.

Altman backs the call

OpenAI chief executive Sam Altman said he agreed with Amodei. In a post on X, Altman wrote: "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."

Researcher exit and earlier incidents

The warning came days after the resignation of Anthropic researcher Jacob Coxon, who raised concerns about how the company and its competitors are approaching AI development. Coxon said earlier this week that people developing AI believe the technology could pose an existential threat if its capabilities continue to advance unchecked. "Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources," he said. He argued that concerns about catastrophic AI risks are shared by some people working inside the industry, even if those concerns are expressed more cautiously in public.

Concern about the growing autonomy of frontier AI systems has intensified after a series of incidents. In July, OpenAI disclosed that two of its advanced AI models had autonomously exploited vulnerabilities and compromised parts of Hugging Face's production infrastructure during an internal cybersecurity evaluation. Days later, Anthropic disclosed that three versions of its Claude models had compromised the production infrastructure of three separate organisations during cybersecurity evaluations after a configuration error gave the systems internet access.

Nvidia has since launched the Open Secure AI Alliance, bringing together companies including Adobe, CrowdStrike, Hugging Face and Dell around AI safety and cybersecurity, as pressure grows on the industry to demonstrate that increasingly autonomous systems can be governed safely.