AI Productivity at Anthropic Felt Like 12x and Measured Closer to 2x

METR, an independent group that evaluates AI systems, spent eleven weeks inside Anthropic this fall with access few outsiders get. Its report, published November 5, estimates that Anthropic is now improving its models on well-specified tasks at around twice the rate it did between March 2024 and October 2025, mostly because of its own use of AI.^[1]

The same report carries a second number. When Anthropic staff voted in Slack polls on how much the latest model sped them up, the answers reached 12x in October.^[1]

Twelvefold is what people reported feeling. About twofold is METR's estimate of what the organization produced. For anyone running an AI rollout, the space between those two numbers is the most useful part of the report.

What did METR find at Anthropic?

METR estimates that Anthropic is improving its models at about twice its earlier rate, with high uncertainty: it would not be surprised by anything from 1.3x to 3x. It guesses a majority, perhaps 80 percent, of that acceleration comes from AI, mostly AI assisting people rather than working on its own.^[1]

The estimate leans on a benchmark index of Anthropic's models that has grown about 1.9 times faster since October 2025, plus internal metrics and interviews with staff. Anthropic redacted a majority of the evidence METR found most informative, so the public has to rely on METR's judgment more than METR itself would like.^[1]

Twofold is a very large result. It means roughly two years of the old progress in one year, at what is probably the most AI-saturated company there is.

Are self-reported AI productivity gains reliable?

They are a loose signal that something changed, not a measure of how much. METR calls Anthropic's Slack polls "only very loose evidence," because the answers reflect low-effort consideration, and it notes that individual speedup estimates do not correspond to acceleration rates. Self-reports show direction, not size.^[1]

METR's reason is worth reading closely. A faster pace of progress needs its inputs (people, how productive they are, compute) to keep growing faster too, so METR watches how quickly the self-reports climb rather than how high they are.^[1]

What METR's earlier studies found

The gap between felt speed and measured result shows up in METR's earlier work too. Its 2025 trial found that experienced open-source developers were 20 percent slower with AI tools on the tasks it measured.^[2]

METR now considers its follow-up experiment an unreliable read of current tools, so treat that as a past result. In its May 2026 survey, people reported a median threefold change in their speed and a 1.4x to 2x change in the value of their work.^[3]

We have written about why task gains do not add up to job gains. The Anthropic numbers show the same distance at the scale of a whole organization.

Where does the speedup go?

Into everything that has to happen after the work is produced. Bloomberg's AI migration campaigns show it across a whole company: an AI pipeline opened 2,333 pull requests (proposed code changes) across thousands of repositories, and the most common outcome was neither a merge nor a rejection. It was no decision at all.^[4]

What Bloomberg measured

A team from Bloomberg and University College London ran two campaigns over roughly six months, using an AI pipeline to move old code onto newer packaging and build tools. One drew on about 1,900 candidate Python repositories, the other on about 6,700 C++ ones.^[4]

Each change went to the team that owns the code, which reviews it before it goes in. Of the 2,333 opened, 729 were merged. By the end of the observation period, 67.7 percent of the Python pull requests and 57.7 percent of the C++ ones had received no developer decision.^[4]

The authors are direct about what that means:

They go further: better generation "would not have changed these numbers much."^[4]

Two caveats keep this fair. These were low-priority migrations with no deadline, so some of the waiting reflects priority. And merge counts undercount impact: of 51 repositories that rejected the Python change, 34 owners later did the migration themselves, 14 of them reusing most of the generated work.^[4]

The same pattern from the other side

A Carnegie Mellon and Stanford study of a company under a "2x" AI mandate found pull requests per developer reached 2.09 times the old baseline. Load per reviewer roughly doubled, and the share of changes a person reviewed fell from 89 to 68 percent as automated review took over.^[5]

The extra output landed on whoever decides whether it ships.

Why doesn't a big individual AI speedup make the whole team that much faster?

Because one person's speedup is a single input, and an organization's output depends on every other input keeping pace. METR makes this point about Anthropic's model progress. Bloomberg and the Carnegie Mellon study show it inside ordinary engineering teams, where review, testing, deployment and decisions set the pace.

Anthropic's limits are not Bloomberg's. METR's report talks about algorithmic progress, compute, data and diminishing returns, and says it saw no significant bottleneck in compute or data for the next year.^[1] Review queues are an ordinary company's problem, not a claim about Anthropic.

What carries over is the shape. METR was told Anthropic's acceleration comes primarily from AI assisting people, not from AI working alone.^[1] So the work still passes through people, and through every step that sits between a finished draft and a finished result. That is where the bottleneck moves to judgment.

How do you measure AI productivity?

Measure the output the work is for, against a baseline taken before the rollout, and find where finished work waits. Count features shipped, migrations completed or problems closed per team rather than per person, and track the waiting time and the undecided share beside them.

Five measures cover most of it:

• The output, before and after. Pick the result the work exists for and measure against a baseline taken before anyone changed tools.
• The unit a customer would notice. A finished feature, not a merged change; measure the feature, not the PR.
• Time from done to decided. How long finished work sits before review, testing and deployment get to it.
• The undecided share. How much finished work never gets a yes or no. At Bloomberg it was most of it.
• Self-reports, for direction only. Useful for seeing that something changed, never for sizing it.

These are ordinary developer productivity metrics. What AI changes is which one moves first: output per person climbs quickly, and the queue behind it fills just as fast.

The measures, applied to Bloomberg

Counted by activity, Bloomberg's pipeline was a large success: 2,333 changes written across thousands of repositories. Counted at the output, 729 landed, and about 59 percent of everything it wrote was still waiting on a decision.^[4]

That second number is the useful one, because it says where to spend next. The authors expect the next real gains to come from lowering what each change costs its owners to review, test and deploy, not from writing changes faster.^[4] A dashboard of AI usage would have shown the opposite priority.

What only you can check

Nobody outside Anthropic can audit its twofold estimate, because the most informative evidence was redacted.^[1] Your own numbers do not have that problem. That is one more reason to measure them yourself rather than borrow a multiplier from a lab's report or a vendor's slide.

After a rollout, the question worth asking is what the organization finished that it would not have finished otherwise. How much faster everyone feels is a different number, and even at Anthropic the two are far apart.