Anthropic's Amodei calls for AI labs to deliberately slow capability growth

Amid increasingly dire warnings from AI researchers about the technology’s dangers, and OpenAI CEO Sam Altman’s own comments that it may be time to “pace” AI development, Anthropic CEO Dario Amodei has laid out what that could actually look like. In a new blog post, Amodei echoed the call to “pace the frontier” and outlined three broad strategies for doing so, saying Anthropic is “unilaterally committing” to one of them — with Altman chiming in to say OpenAI will follow suit.

The debate over AI safety intensified this week after researcher Jacob Coxon wrote that he’s resigning from Anthropic over concerns that leading AI companies are “gambling with our lives” while the people building the technology “earnestly believe it could kill us all by the end of the decade,” a claim he said is repeated by others at Anthropic. Amodei’s post doesn’t explicitly mention Coxon’s resignation, but he wrote that two things convinced him it’s time for a more cautious approach: the OpenAI-Hugging Face hack, and the fact that “AI has been advancing drastically faster” in recent months, particularly its “growing ability to build the next generation of AI.” “We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”

Amodei’s first proposed step involves “embedded evaluators” from third-party organizations like METR, who could verify AI companies are following their pacing and safety commitments and ensure safety incidents get reported — comparing them to bank regulators embedded with employees. He said this is “something Anthropic is unilaterally committing to,” giving evaluators badges, desks, and laptops with access “mostly comparable to what internal risk assessment teams have.” Altman called it a “good idea” and said OpenAI would do the same.

Next, Amodei called for leading AI companies “within democratic countries” to coordinate on common safety standards and limits on the rate of unchecked AI progress, acknowledging this could raise antitrust concerns and suggesting the U.S. government could “mediate or at least enable” such discussions. He also argued that steps like restricting chip and semiconductor-equipment sales to Chinese companies, alongside cracking down on model distillation, could “slow China’s progress enough to widen America’s lead significantly over the next 3-5 years.” Lastly, he called for “global coordination,” including “cooperation with China,” while admitting “stark limits on what can be achieved” — suggesting narrow agreements might still be possible, such as prohibiting AI use in producing biological weapons.

Some AI boosters have criticized Amodei as a doomer whose comments feed the current AI backlash; he responded that he’s tried to offer a “balanced” perspective and that the backlash reflects a broader “crisis of trust” in tech companies and government. Others, including journalist Brian Merchant, have been skeptical of the apocalyptic framing, with Merchant writing he has yet to see “a credible, step-by-step documentation” of how AI could move from self-improvement to killing humanity, and suggesting proposals like Amodei’s “would likely only wind up serving Anthropic and OpenAI.”

Why it matters

Amodei’s proposal turns AI safety rhetoric into a concrete governance mechanism — third-party evaluators inside frontier labs — that could become a template regulators point to, even as critics see it as a way for incumbents to shape the rules in their favor.