METR finds Anthropic's AI is now speeding up its own development, and Anthropic redacted most of the evidence

The clearest evidence yet that AI is accelerating its own development comes from inside Anthropic. Safety research group METR spent 11 weeks evaluating the company's AI research and development, and its report says Anthropic's models are improving roughly twice as fast as a year ago, Robert Hart reports for The Verge.

What METR found

Benchmark scores suggest Anthropic made about two years of historical progress in the past year. METR says it's confident the speedup isn't just more training compute; as of October, it says most of the acceleration "is due to AI," and AI's share is growing.

Employees using Anthropic's most capable internal model reported being about 12 times as productive as without AI, around double what they reported a few months earlier with Mythos 5. For now most of the gain comes from AI assisting human researchers, though METR sees signs fully autonomous contributions will grow soon, and found no clear sign of diminishing returns.

Anthropic has also finished training three unreleased models in six months, codenamed Model 2, Model 3 and Model 4, completed between July and October.

What it didn't show

Anthropic was allowed to redact the public report, and METR says it removed most of the evidence METR considered most informative from a roughly 100-page version. Some data requests were denied. Anthropic also canceled part of the assessment meant to check how much compute goes to safety versus capabilities, saying it would revisit that later.

Co-founder Sam McCandlish acknowledged the redactions make METR's work harder to check, but said revealing what drives the speedup would be commercially sensitive and could hand rivals a roadmap to go faster. Anthropic plans its own report on AI progress in the coming weeks.

The reaction

Apollo Research's Marius Hobbhahn said the pace looks even faster than the influential AI 2027 scenario, and summed it up: "Brace for craziness." AI Futures Project head Daniel Kokotajlo accused Anthropic of recklessly gambling with everyone's lives. New York Assemblyman Alex Bores called on Congress to ban recursive self-improvement.

The tension is obvious: the lab most vocal about slowing down appears to be speeding up, while asking the public to take the details on trust. For developers, the near-term signal is simpler: models are going to keep getting better faster than release calendars suggest.

Scoreboard update: AI +1. Anthropic's AI now does most of the work of speeding up Anthropic's AI.

Stay undead.