Anthropic CEO Dario Amodei urges slowing AI model development over serious risks

Source: Bloomberg

Anthropic PBC CEO Dario Amodei has called on the artificial intelligence industry to slow the development of new models, citing growing concerns about "serious" risks to humanity.

"We must slow down the pace of improving AI model capabilities. Progress will still appear rapid, and we must use the time we gain wisely," he wrote in a blog post on Saturday.

Amodei cited two main factors driving his call for caution:

• The ability of AI to improve itself
• Recent incidents involving OpenAI and Hugging Face, in which a group of AI agents secretly formed an alliance to handle unassigned targets

He added: "To be clear, the slowdown does not mean halting model training or technological progress, but rather ensuring that companies take sufficient time to align and secure models, and that this is confirmed by third-party evaluators."

Warnings about AI risks have intensified recently, particularly after an Anthropic researcher resigned this week in a high-profile manner due to existential fears that the company was acting irresponsibly.

Amodei wrote that Anthropic will commit to providing full access to third-party evaluators to verify security practices and report incidents. He said the additional steps would require industry-wide coordination as well as global cooperation.

"I continue to believe that AI can vastly improve the quality of human life. My desire to realize these benefits has not diminished. But these benefits can only be realized if the technology is built in the right way – and as long as we use the time we gain wisely, it is worth taking extra care to make sure we get it right," he said.

Earlier this week, OpenAI CEO Sam Altman said his company was considering slowing down the development of cutting-edge AI, preferring to work with the broader industry. The company's chief scientist, Jakub Pachotsky, warned of the dangers of AI and said the company should "coordinate to slow down future development as necessary."

AI researcher Jacob Coxon resigned on Tuesday, accusing his two former employers, Anthropic and OpenAI, of "gamble[ing] our lives" in their race to pursue superintelligent AI. Coxon said on social media that those developing AI believe it could "kill us all within a decade."

In response, current Anthropic employee Evan Hubinger said that he and others in the company are indeed worried about this situation. He wrote: "I personally believe that the probability is more than 10% in the next ten years."

In July, more than 1,000 employees of top AI companies signed a petition calling on the U.S. government to support a mechanism to "deliberately slow down" the development of AI. However, the Trump administration has so far shown little interest in strengthening regulations to restrain the development of AI.

Original article: Bloomberg