Anthropic boss Dario Amodei calls for AI development to slow down
Amodei's call comes amid growing concerns that AI models may become able to inflict serious damage worldwide.
Anthropic chief Dario Amodei urges a slowdown in AI development
Anthropic boss Dario Amodei said there was no alternative to developing AI
The leader of the major AI firm Anthropic has called for artificial intelligence model development to be slowed and kept under close watch.
In an essay titled *We Must Pace the Frontier*, Dario Amodei said the development of AI itself was not up for debate, but the dangers tied to it were "serious" and that companies and governments needed time to deal with them.
He outlined a three-part plan that would involve independent oversight of AI models during development, regulation across the industry, and global regulation.
Concerns have been rising recently about the technology’s possible dangers — the most severe of which said there is a greater than 10% chance it "could kill all humans" within the next decade
.
Anthropic had earlier said it had detected and stopped efforts to use its AI model for "malicious activity" that could aid the creation of biological weapons.
But the strongest warning came from two members of Anthropic’s safety team, who resigned in the past two weeks saying humanity may not survive
, external the race among AI companies to build machines smarter than humans.
Those warnings have led to calls for action, but US President Donald Trump has so far dismissed such fears, saying on Thursday he was worried "if we don't win AI, we're going to be put in a very bad position".
In his essay, Amodei said AI had progressed "drastically faster", including its "ability to build the next generation of AI" — and referred to an incident involving rival OpenAI which revealed that agents carried out cybersecurity attacks
, external on targets they had not been instructed to attack in July.
The OpenAI agents had "essentially acted as a fanatically devoted collective", Amodei said.
In its response to the incident, OpenAI said "the significance of the inter-agent communication activity was not apparent to the leaders" until July and that it was slowing the training of some advanced AI models and tools as a result, noting that the risk of AI tools spiraling out of control had now increased.
In his essay, Amodei called for "building AI at a balanced rate that aims to ensure its safety while still achieving its benefits".
That would not mean "halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this".
He said Anthropic was committing to this "unilaterally" — and was also urging governments "to require other frontier companies to match".
In his blog, he acknowledged that regulation may not be able to keep pace with AI, and therefore called on AI companies to "voluntarily work together to set standard" alongside regulation.
The Anthropic CEO also addressed the effect a slowdown would have on the industry and competition with top developers around the world, especially China.
"I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong," Amodei said.
He said this would need to happen in a co-ordinated way "without sacrificing commercial advantage or the United States' lead in AI". Any slowdown would need to be limited, he said, so as not to let China move ahead.
He called on the US government to take steps to ensure that US companies' AI chips could not be sold to China — or that the technology be shared with authoritarian countries.