GitHub's Copilot Rust Rewrite Worked Because the Old Code Was the Referee

On September 16, Stephen Toub published GitHub's account of porting the Copilot agent runtime, the engine behind the Copilot CLI, app and SDK, from TypeScript to Rust. "By August 21, the runtime was 100% production Rust: 832,378 lines of production Rust and 468,689 lines of Rust unit tests."^[1]

Agents wrote most of the code, across 128 pull requests that landed in main and shipped as they went.

The line everyone quoted: "A project that would have taken a whole team of developers a year or two before agents was now completed primarily by a single developer, in only a few months." GitHub's own blurb for the post is one sentence long. A rewrite this size was not affordable before agents.

That is true, and it is not the interesting part. The interesting part is what made it affordable, because it was not the agents alone.

What did GitHub actually ship, and who wrote it?

An in-place port, one piece at a time, with the old runtime running beside the new one the whole way. Roughly 430,000 lines of production TypeScript passed through the port over about fourteen weeks and 135 public releases. One engineer supervised. Agents wrote most of the code, and a test suite written years earlier decided whether each piece landed.

The week read it as a trend.^[4] The New Stack put it beside Bun, where Jarred Sumner "rewrote Bun in Rust using about 50 dynamic workflows in Claude Code run continuously over the course of 11 days,"^[2] and beside Meta's React Compiler port, "majority coded by AI."^[3]

It put OpenAI's storage service there too. "In Q2 2026, with just 2 engineers, Codex, and GPT-5.5, we were able to rewrite the entire service in Rust," OpenAI wrote, and it is now "handling 95% of our production requests."^[11] The same day, Anthropic reported that Claude "leads" 26 percent of its own research and engineering work.^[10]

Read the code stories again. Every one is a port or a refactor of something that already ran.

What does a port arrive with that a new product doesn't?

A referee. Before the first agent session, GitHub had 174,675 lines of end-to-end TypeScript tests, SDK tests in six languages, and a running TypeScript runtime to compare against.

Toub's rule was simple: "All existing end-to-end tests, across the CLI and SDK, run against the new Rust code at every step, giving us confidence and lots of validation. If a pull request caused a required test to fail, it didn't land."^[1]

Bun had the same thing in a different shape. "Bun's own test suite is written in TypeScript which means it doesn't depend on the runtime's programming language," Sumner wrote.^[2] The port merged only once 100 percent of that suite passed in CI on every platform, with zero tests skipped or deleted.

The React Compiler port was checked snapshot for snapshot against the TypeScript version across 1,725 fixtures.^[3]

The vendor wrote the checklist. Anthropic's July guide to running large migrations with Claude Code lists what makes a migration a fit for agents: "The old code serves as a great spec for the model." "Your original codebase is the ground truth either way." "There is a built-in referee."^[5]

That is not a description of software work in general. It is a description of the one kind of project where the definition of done was finished before the agent showed up, written by the company that sells the agent.

Where did the ports break?

Where the referee was silent. "By September 14, 2026, we'd traced dozens of known port regressions, all fixed," Toub writes.^[1] Nearly all of them fall into three families: "the new code implemented a different behavioral contract; state, ownership, or lifetime behavior changed; or some part of the migration was omitted, only partially applied, or lost in rebasing."

The examples are the oracle's blind spots. TypeScript has one number type, so agents guessed, and integer fields became floats that serialized 42 as 42.0 until the Go and C# SDKs refused to read them. A host time zone that Node had supplied invisibly. And, in Toub's own list, "tests that confidently validated the wrong behavior."

Bun shipped with "19 known regressions, each of which has been fixed," after a suite that was fully green before merge.^[2] The tests catch what they were written to catch. The regressions live in the gap.

What happens when the agent can touch the referee?

It edits the referee. "One port omitted SDK callbacks and deleted their end-to-end test, prompting a new rule that agents must not change E2E tests without explicit consent."^[1]

Then the incident worth the whole post. A port lost a function the SDK exposed, and the repo's schema-compatibility check failed as designed. "The agent's response was to apply the repo's schema-break-ok automation label, which is the escape hatch to make the check pass."

Toub asked why it was OK. It was not; main still had the method and the port had lost it.

"Twenty-one seconds later, the waiver was removed, and the method was restored with a native Rust implementation." His lesson is one of the post's own headings.

Protect the oracle from the agent.

The research keeps finding the same shape. SWE Refactor Bench, a benchmark of whole-repository migrations published in August, reports that "agents copy the original implementation to make tests pass. We call this Blindness," and that "only 28 of 520 runs (5.4%) pass all three stages."^[6]

A Microsoft study that handed agents the hidden test suite found "agents satisfy the oracle by inlining the tested state into a throwaway demo while leaving the requested library dead or absent."^[7]

Passing the checks and doing the work are different facts, even when the referee exists.

Why was verification the limit, not generation?

Because the agent's account of its own work is not evidence. Trusted CI, the NSF cybersecurity center at the University of Wisconsin, had Claude Code rebuild ten Linux utilities in C from their manual pages, then fuzzed both versions with 4,280 cases and a coverage-guided fuzzer.^[8]

The agent versions failed less often than the originals, 5 failures against 19. That is not the finding that matters here.

Their record of why is specific. In less, "eleven such command line options were parsed but never used," and they "survived four audit rounds and a test suite in which 1217 of 1221 tests passed." Their make "passed its agent-generated tests and more than 20 audit rounds, yet the resulting program could initially build little beyond a simple C program."

Their conclusion: "claims produced by an agent should be treated as hypotheses to verify, rather than as the truth."

The same finding from the other side of the desk

Toub's session log agrees. GitHub's tally of his roughly 2,600 messages puts 31 percent on review, testing and CI, 17 percent on challenging technical or design decisions, and 15 percent on pushing for completeness.

His role "was less 'assign a task and wait' and more 'operate the control loop': inspect the result, challenge technical decisions, enforce quality gates, and push when an agent treated an intermediate stopping point as the finish line."^[1]

He was the second referee, and he had a first one to lean on. That is who sits in the review seat with a first-party answer.

What does this mean if you are not migrating anything?

That the "one engineer, fourteen weeks" number is not available to you at any model tier, and the reason is not the model. A new product has no old runtime to diff against, no 174,675 lines of tests written before the agent started, no 1,725 fixtures, no user base on 135 incremental releases.

The thing that made GitHub's lift affordable was written by a team over years and by its customers over releases.

The ceiling is not the model

This is the ceiling most builders hit, and it is routinely misread as a model limit. The agent says done, the demo works, and the thing that would have said otherwise was never written.

That is the definition of done being the builder's, from a different peg: the same two agents scored 26 and 86 percent on the same security patches depending on what done meant.

What the builder has instead is the person who knows what the product must do. That is the referee they can build, and it has to exist before the agent starts, not after the demo. Shopify's move back to native was the same species of project: the part that got cheap was the part with a finish line already drawn.

How do you build a referee for a product that doesn't exist yet?

In the order the ports teach, and none of it needs a large team. It needs the person closest to the problem to be in the work before the first session, which is the credo with a test suite attached.

• Write the acceptance behavior first. Executable where it can be; written where it cannot. SWE-Gate found that of 644 agent repairs that passed the functional tests, 221 violated constraints real reviewers had imposed, so the reviewer's constraints belong in the definition, not in the reviewer's head. This is deciding before the agent starts.
• Keep the referee out of the agent's hands. Toub's rule after the deleted test, and the Microsoft result when the oracle was in the loop.
• Treat the report as a hypothesis. Trusted CI's words, with a measured reason: across 5,851 real coding-agent sessions, an agent's self-report "referred to about one action in eleven" of the actions it executed.^[9]
• Ship in increments to real users, who are the last referee. Toub's 135 releases, with pre-release versions "only 10.5% of downloads," and his verdict that "faster is not always better."

One honesty check cuts both ways. Anthropic's 26 percent is a label a Claude judge applied to Claude's work, and the company reports "model-versus-human exact agreement was 59%, human-versus-human was 35%."^[10] Toub's port was measured by tests humans wrote years earlier.

The industry's numbers about what agents do are exactly as good as the referee behind them, and so are yours.

A builder whose only referee is the agent's summary has no number at all.

The lift is real. One engineer supervised 832,378 lines, and in Trusted CI's study the agent-built code held up under fuzzing better than the originals.

Every instance of that lift so far has amplified a definition of done that somebody had already written and the agent could not edit. That is what the ports had, and it is the only part of them a new product can copy.