Anthropic Wants a 'Brake Pedal' for AI Before It's Too Late: Claude Already Writes 80% of Its Own Code

80% of the code merged into Anthropic's codebase is now written by Claude. And the company's co-founder says the industry has zero mechanism to slow down.

Anthropic warns of recursive self-improvement in AI

In a blog post from The Anthropic Institute, co-founder Jack Clark and institute leader Marina Favaro warned that "full recursive self-improvement" is closer than expected. That's the threshold where an AI system autonomously designs and trains its own successor, no humans required.

Anthropic revealed that engineers now merge eight times more code per day than in 2024, because Claude writes most of it. Clark used a car metaphor on CNN: the AI industry has a gas pedal but no brake pedal. And we might want that option soon.

Cold War parallels and a call for cooperation

Clark drew explicit comparisons to nuclear arms control during the Cold War. Rival nations found ways to stabilize the arms race under extreme tension, he argued, and competing AI labs should do the same.

Anthropic is calling on companies to consider slowing or pausing frontier AI development. The company also plans to convene policymakers, researchers, and civil society in the coming months to address the questions that recursive self-improvement raises.

Safety warning or pre-IPO marketing?

Here's where it gets interesting. Anthropic filed its confidential IPO paperwork on June 1, just days before this public alarm. The company sits at a $965 billion valuation after raising $65 billion in its Series H round.

Critics argue that amplifying existential risk could encourage regulations that conveniently block smaller competitors. Supporters counter that Anthropic is one of the few labs willing to say the quiet part loud. Both sides have a point.

A rocket builder asking for a speed limit

Unilateral slowdowns risk handing the advantage to less cautious players, particularly in China. But racing ahead without safety mechanisms is a bet that alignment will keep pace with capability. Anthropic holds a genuinely awkward position: a rocket builder demanding a speed limit. Whether that's principled or strategic is for you to decide.