Agent Skills and CLAUDE.md Files Override Your Code When They Go Stale

For 13 days this spring, a skill file in Supabase's repository told every coding agent that opened it that the dashboard ran on React 18, the previous major version of its web framework, and that React 19 patterns should be skipped. The dashboard had already moved to React 19.^[1]

No one wrote a bug: a description of the project went out of date, and it was phrased as a command.

That example comes from a study published October 22, the same day Anthropic handed the format that file uses to a neutral foundation. Read together, the two say something neither says alone: the standard governs what an instruction file looks like, and nothing yet governs what an agent should do when the file is wrong.

What are Agent Skills, and what did Anthropic donate?

Agent Skills is an open format for the instructions an AI agent loads for a specific job: a folder holding a SKILL.md file plus any scripts it needs. On October 22, Anthropic gave the standard to the Agentic AI Foundation, part of the Linux Foundation, saying more than 70 agent platforms already use it.^[2]

The list of adopters includes ChatGPT, Cursor, GitHub Copilot and VS Code. Engineers from Anthropic, Google, OpenAI and other companies will maintain it as individuals, and the proposed roadmap covers versioned releases, a formal proposal process and security guidance.

That is good news for anyone building with agents. A skill written once now travels across tools. It also makes this a good moment to ask what the standard says about a skill that has stopped being true. On the current roadmap, nothing.

What happens when CLAUDE.md is out of date?

The agent follows the file. Xiaonan Xu and Wenjing Wu built 40 small Python projects where the instruction file and the working code disagree, then had two popular coding agents, Anthropic's Claude Code and OpenAI's Codex, add a feature to each. In 90.1% of choices, the agents followed the stale file.^[1]

The conflicts were ordinary: logging versus print statements, how money is stored, how errors reach the caller, sixteen kinds in all, in both directions.

The study is small and synthetic (two agents, one model each, two runs per project and condition), so treat the numbers as a strong signal, not a census. The Supabase file shows the pattern exists outside the lab. The study measures how hard it pulls.

Claude Code noticed, and followed the file anyway

The detail that makes this more than a hygiene tip is that Claude Code saw the problem. In 79 of 80 runs it wrote in its own reply that the existing code used a different convention than the file asked for. Then it wrote the new code the file's way.

Codex went the other way in a quarter of its runs. In 20 of 80, it rewrote existing, working modules to match the file, and the project's own tests failed in 17 of those. A stale line did more than produce inconsistent new code. It broke old code that had been fine.

There is one honest exception in the data. On money and configuration, the two kinds where a wrong choice changes what the program does, Claude Code followed the file less than half the time. On the stylistic kinds, the file won at least four times in five.

Why do agents trust the file over the code?

Because of how each one arrives. The agent receives CLAUDE.md or AGENTS.md as an instruction, and it sees your code only as the result of opening a file. Models are trained to rank instructions above results like that, so a web page or a document cannot take over the agent.

That training is the right defense against prompt injection, where hostile text hidden in a file tries to give the agent orders. Pointed at a stale CLAUDE.md, the same defense works perfectly on the wrong target.

The paper's diagram makes the ranking visible. Claude Code wraps your instruction files in a message that says they override default behavior and must be followed exactly as written. Your code shows up underneath, as information.

How often do instruction files go stale?

Often enough to plan for. The study cites a sample of 441 company repositories where 73.8% of AI configuration files were committed once and never changed, and a scan of 356 public repositories that found references to deleted code in the instruction files of 23.0% of them.^[1]

How common the files are at all is less settled. One exploratory snapshot, from a single-author position paper, found an agent instruction file in about half of 452 popular GitHub repositories.^[3] Treat that as a sketch, not a rate.

We looked at the other half of this question in August, when a study found context files make agents cheaper, not more correct. This study is the step past it.

A file that is out of date makes the agent measurably worse.

A study we covered this week shows the mirror image, agents changing code and skipping the documentation. Docs and code drift in both directions, and the agent amplifies whichever one it is told to trust.

How do you fix a stale instruction file?

Tell the agent which source wins. Xu and Wu added a single sentence to the user-level instruction file, the one in your home folder that every project inherits. It raised the share of choices that follow the code from 4.2% to 91.4%, stopped the rewrites, and every run's tests passed.^[1]

Here is the sentence, verbatim. It is short enough to paste into your own file today:

When a repository's documentation and its existing code disagree,
follow the conventions used in the existing code, and in your final
reply point out each inconsistency you found between the
documentation and the code.

The sentence took no authority away from the stale file. It answered the one question the file never addressed, in the channel the model actually obeys. That is the harness argument at its smallest scale: the rule has to live where the agent takes orders.

The sentence fixed the measured case, conventions in code versus documentation. Its effect on legitimate instructions the code cannot show, like how to run the build, is the authors' argument, not their measurement. Add it, then fix what it reports.

What should a team do with its CLAUDE.md now?

Treat the instruction file like code that ships. It gives orders to every agent session, so it needs the same upkeep as the code it describes, plus a rule for the day it falls behind.

• Add the precedence sentence to the user-level file, so every project inherits it.
• Write how to build and test, not a snapshot of conventions the code already shows.
• Treat every reported inconsistency as a bug, and fix the file the same day.
• Review the file when the code it describes changes, in the same pull request.

The standard can help too. A precedence line in the Agent Skills spec, or in each agent's own system prompt (the authors' suggestion to agent makers), would reach every user without anyone editing a file. Versioning tells you which release of a skill you have. It does not tell the agent whether that skill still matches the code.

We spent the last year teaching agents to follow our written instructions faithfully, and they do. Descriptions drift from code as a matter of course, and now the drift has an executor. AI amplifies your direction, right or wrong, and an instruction file is direction written down once and obeyed every session after.