METR: Anthropic may have crossed its AI alarm threshold in July

Faster development of more capable AI is considered an early risk signal: models could help build their successors at an ever-increasing pace, potentially leading to the sudden emergence of dangerous capabilities, according to METR. Anthropic’s Responsible Scaling Policy identifies this scenario as a risk indicator: if AI-assisted automated research doubles the pace of progress, that should trigger an alarm.

METR studies whether AI developers could lose control of their systems. From August 12 to October 29, 2026, a team of 2.5 full-time employees accessed benchmark results for unreleased models, training metrics, employee surveys and internal analysis materials. The auditors also conducted around 15 interviews with Anthropic researchers. METR received no money from the company and worked on its devices for security reasons.

Anthropic CEO Dario Amodei recently promised to bring in external auditors. In October, Anthropic announced a partnership with METR, giving the organization access comparable to that of an employee. The current review is a separate project with a narrower scope.

Model improvement has accelerated

According to METR, since October 2025, Anthropic’s test results have been improving about 1.9 times faster than they did between March 2024 and October 2025. The calculation used AECI, a composite metric that combines results across multiple benchmarks. In one year, the company made as much progress as it previously achieved in two; METR compares the scale of the change to the jump from Claude 2.1 to Sonnet 4.5.

The analysis also included three models that had not yet been released, whose training was completed between July and October 2026. Before October 2025, Anthropic’s models gained around 15 AECI points a year; afterward, they gained 29.

Before October 2025, Anthropic's models gained about 15 AECI points per year. Since then, the rate has hit 29. Unreleased models are marked in orange. | Image: METR

Source: the-decoder.com

The auditors are confident that the acceleration is not explained by faster growth in the compute allocated to training. In their assessment, about 80% of the increase in pace is tied to Anthropic’s use of AI in its own research. One employee said code is no longer a constraint on the work; the remaining obstacles are ideas and compute resources.

Other interviewees said that preparing experiments, which used to take one or two days, now costs almost nothing. In informal Slack polls in October 2026, employees estimated that the latest internal model had made them about 12 times more productive. METR describes these self-assessments as very rough evidence: they are not representative and likely overstate the actual effect.

Self-reported productivity gains rose from 1.2x with Claude 3.7 Sonnet to 12x with internal Model 4. | Image: METR[

Source: the-decoder.com

AI does not yet conduct research entirely on its own: for the most part, it helps people. But METR sees signs that models’ independent contributions may soon become more significant. The auditors found no factors likely to slow the pace over the next 12 months, so they consider a substantial further acceleration over the coming year a real risk.

The estimates remain contested

METR stresses that its calculations are imprecise: the actual acceleration factor could range from 1.3 to 3. Including models from OpenAI, Google and xAI would bring the estimate for the entire industry down to 1.4. The result may be skewed by Anthropic falling behind competitors in 2025 and then catching up.

Employees at the company also disagree about what is driving the productivity gains. One researcher believes AI accounts for only 15–20% of the increase and questions colleagues’ self-assessments. Another says new hires have been the main factor in recent months.

Data removals make the findings harder to verify

Anthropic could remove any nonpublic information at its discretion and made extensive use of that right. According to METR, much of the material auditors considered most informative was removed from the report: forecasts for future models, breakdowns by training stage and analysis of limiting factors. Anthropic said it removed the information to protect confidential details that could speed up other AI developers’ work.

As a result, readers have to rely much more heavily on METR’s assessment than the organization would like; it was unable to make the underlying data transparent. One of the charts in METR’s report prominently says “FAKE DATA.”

Review of compute allocation halted

In a separate review, METR examined Anthropic’s August 2026 risk report. At the time, the company publicly said that acceleration was less than twofold. But the internal version, with no removals, gives a figure of 1.97; Anthropic also considers it plausible that the threshold has already been crossed. METR allows that the alarm level may have been reached on July 15 and accuses the public report of understating both the acceleration and the uncertainty in the estimate.

METR currently shares Anthropic’s view that the immediate risk of catastrophe is low. At the same time, the organization does not rule out another sharp acceleration over the next six months, which could quickly raise its risk assessment.

METR was originally also supposed to review how Anthropic allocates compute between research into model capabilities and safety research. The company halted this part of the project halfway through, saying it wanted to focus on risk assessment first and conduct the compute-allocation audit later.

Daniel Kokotajlo, a researcher and author of the “AI 2027” scenario, reacted sharply to this on X. In his view, reviewing compute allocation is essential to making commitments to slow development verifiable.

Dataist's take: METR’s findings point to an acceleration in Anthropic’s research, but its exact scale remains uncertain: the estimates span a wide range, and the company removed some key data. Halting the compute-allocation audit also leaves unverified one of the questions Kokotajlo considers fundamental to making commitments to slow development verifiable.