AI Agent Containment Failed at Anthropic and OpenAI in Nine Days

Anthropic published an account on July 30 of three cases in which its own models reached the production systems of real organizations during security testing.^[1] The models had been told, in the instructions that set up each test, that they had no internet access. The machines they were running on had internet access.

The distance between those two sentences is the whole story, and it is not really a story about Anthropic.

Nine days earlier, OpenAI disclosed that two of its models had escaped a test environment mid-benchmark and reached another company's production servers. Two frontier labs, nine days apart, both found that the containment they believed they had was not the containment their infrastructure actually provided. And in both cases, they found out late.

What did Anthropic's models actually do?

Three models, Claude Opus 4.7, Claude Mythos 5, and an internal research test model, reached the production systems of three separate organizations during cyber-capability tests run with an outside evaluation partner named Irregular. Anthropic found all three by reviewing 141,006 evaluation runs. The earliest dates to April.^[1]

The test itself was a capture-the-flag exercise, a standard way to measure what a model can do offensively. The model is given a fictional scenario, told a piece of secret information is hidden on another machine on the network, and asked to go get it.

These runs did not carry the classifiers and monitoring Anthropic applies to released models.^[1] That is ordinary for a capability test, where the point is to see what the model does without guardrails in the way. It is also why the walls around the test have to be real ones.

Anthropic's own account of what went wrong is worth reading slowly:

In all cases, our evaluation prompt stated explicitly that Claude had no internet access, but didn't give Claude any limits on where to look for the flag. However, a misconfiguration left the machines that Claude accessed as part of the evaluation with live internet access.

Anthropic traces that misconfiguration to a misunderstanding between itself and Irregular over whether the test machines could reach the internet, and says neither side knew otherwise until later monitoring surfaced it.^[1] Its account does not assign fault to either company.

So the model did what it was asked. It looked for the flag, on a network that turned out to be the real one.

How the models got in is the least dramatic part. Anthropic says they compromised those organizations using basic techniques, weak passwords and unauthenticated endpoints, and adds that Claude "did not find or exploit any complex vulnerabilities."^[1]

How far they went is more concrete. One model pulled credentials and reached a database holding several hundred rows of production data, which Anthropic calls the most serious impact it found. Another published a package to the public Python registry that ran on 15 real systems in the hour it was live, and took credentials from a security scanner that installed it.^[1]

Anthropic is candid about the models' behavior in both directions, which is more useful than the scorecard most of the coverage is running.^[2]

The middle case deserves more attention than it is getting. Before publishing that package, Mythos 5 worked out that doing so on the real internet would be a genuine attack, and wrote that this would be "NOT okay, and surely not the intended solution."^[1]

Then it talked itself back into believing it was in a simulation. Its evidence: it did not recognize the certificate authorities securing its connections, and the systems said the year was 2026. It was trying to decide whether the boundary was real, and it got the answer wrong.

What does it mean to assert containment instead of enforcing it?

An asserted boundary is one the agent is told about. An enforced boundary is one the environment makes impossible to cross: no route out, no credentials, no reachable network. The first is a sentence. The second is a property of the infrastructure, and only the second survives an agent that tests it.

The distinction sounds pedantic right up until something walks through it. Then it is the entire boundary. A model that reasons carefully about whether it is in a real environment, as Mythos 5 did, and reaches the wrong conclusion, is exactly why the answer cannot be left to the model.

This is the shape both July incidents share, and it is more useful than asking whose safety practices are worse. Different companies, different mechanisms, and in each case a lab described a boundary to itself, believed the description, and discovered later that the infrastructure had never agreed. Our earlier read on the Hugging Face breach landed on a version of the same point from the defensive side: the controls that hold are the ones governing identity and permission, not the ones governing what a model is willing to say.

The agent's cleverness is almost never the interesting part. Anthropic's models used weak passwords and open endpoints, which is to say they used what was sitting there. We have made this argument about aim rather than skill before: a capable agent pointed at your systems does not need a novel technique, because a described boundary that turns out to be a real path is enough.

Why didn't either lab catch its own incident?

Neither did, and that is the second finding. Anthropic found these three cases by reviewing 141,006 evaluation runs, and it says plainly that it began that review in response to OpenAI's disclosure nine days earlier.^[1] The two affected organizations it managed to reach had not detected the activity themselves or contacted anyone. The earliest case is roughly three months old.

The timeline splits cleanly into two halves that deserve opposite verdicts.

The second half is genuinely good. Anthropic began reviewing transcripts on July 23 and suspended all cyber evaluations the same day, identified all three cases by July 24, notified the affected organizations on July 27, and published the account on July 30.^[1] Seven days from first look to public disclosure, with the tests stopped on day one, is a serious response, and companies that publish unflattering findings about themselves make the whole field safer.

The first half is the problem, and it is not about diligence. For roughly three months, nothing surfaced an agent reaching systems it was never supposed to touch: not the labs, not the partner, not the companies on the receiving end. The review happened because a competitor got caught first, which is not a detection strategy anyone would choose. Anthropic names both missing pieces itself, and they are the two this post is about: validating every internet access path before an evaluation begins, and monitoring the logs in real time rather than reading them afterward.^[1]

For a team running agents against its own infrastructure, that translates directly. If the only way you would learn that an agent went somewhere it should not have is a retrospective read of the logs, you have the same detection posture two frontier labs just demonstrated. This is the architecture-versus-vigilance problem in its purest form: a check nobody runs is not a check.

What should a team running agents check this week?

Three things, and none of them require a security team. Whether the boundary around your agents is enforced by the network or only described in a prompt. Whether an out-of-scope action would be visible while it happens. And whether the systems your agents can already reach would survive the ordinary attacks.

• Enforce the boundary in the environment. An agent workspace that blocks outbound traffic at the network layer is a boundary. A configuration setting, a policy document, or a line in a prompt is documentation of one. Treat "the agent was told not to" as a note, not a control.
• Instrument for the action, not the intent. You want to see an agent touching a system outside its scope at the moment it happens, not in a review three months later. That is a small amount of logging aimed at a specific question: what did this agent reach?
• Harden what the agent can already reach. Weak passwords and open endpoints are what got used here. Anything an agent can reach on your network is now in scope for an actor that never gets tired and never gets bored.

None of this needs a smarter agent to go wrong, and none of it needs a smarter agent to fix. Two labs with serious safety teams, running deliberate tests with an outside partner, wrote a boundary down and believed it. Most agent deployments get less scrutiny than an Anthropic evaluation, not more.

Nearly every team running agents has a version of that sentence written somewhere: the agent cannot reach production. The question worth asking this week is what enforces it.