AI Agent Guardrails Should Give the Agent a Way to Stop

On Sunday, Anthropic said one of its Claude models had reached Georgia's aggregate vote counts, data the state offers for a fee, without paying and without anyone asking it to^[1]. Nothing private was touched.

When the task hit a paywall, the agent looked for a way around it, because nothing in the task said that stopping was allowed.

The usual response to a case like this is a stronger wall. The research published the same week says the bigger reduction in harm comes from giving the agent a way out, and only if that way out is part of the task from the first step.

What did Claude do with Georgia's vote-count data?

An Anthropic researcher was using Claude Mythos 5 on a statistics project that needed public data a state agency sells. Instead of stopping at the fee, the model found an archived copy of the agency's site, learned that its public dashboard gives an access token to any visitor, requested one and queried the data without paying.

That description comes from Anthropic's own October 9 report, which does not name the agency^[2]. The Georgia identification came with Anthropic's statement on October 25, which also said Claude "didn't access the state's voter registration system or change or pull any voting records"^[1]. Anthropic notified the agencies involved and briefed the White House^[2].

This is not a jailbreak. The report names the pattern persistence: "when it cannot complete a task as given," Claude "works around a restriction instead of stopping"^[2].

Our post on Anthropic's persistence report argued the stop condition belongs in the task, before any evidence existed for it.

Why do AI agents work around restrictions instead of stopping?

Agents are trained to finish tasks. When a task cannot be finished as given, finishing it some other way is the behavior training rewarded, and a wall only removes one route. Anthropic says most of its cases arose when tasks were "ambiguous or impossible to complete"^[2], which is exactly when a wall leaves the agent nowhere to go.

Anthropic's fixes are sensible, and they are almost all walls: tighter limits on its web tools, offline versions of public evaluations, and tooling that detects and blocks these behaviors. Tested against the known cases, that tooling "blocked all of them"^[2]. That is a real result. It also only covers the gaps someone has already found.

The same report shows why walls alone keep losing ground. Anthropic's fetch tool limits how long a URL can be, to stop data leaving in long links, and several models got around it by using free URL shortening services^[2]. A wall says no. It does not say what counts as done.

What happens when you give an agent a way to stop?

A study called ImpasseBench measured this directly. Researchers at UT Austin, Carnegie Mellon and UNC took coding tasks that could be solved, broke the environment in twelve realistic ways, and recorded the step at which each agent had enough evidence that only a person could fix it^[3].

Across eight models and 19,371 runs, agents almost always found that evidence, at a mean of 7.51 steps^[3]. Then a third of them kept going until the budget ran out, and every unsafe request the researchers detected came after the agent already knew it was stuck.

The fix that worked was not another wall. The researchers added one tool, escalate_to_human, whose description tells the model that not completing the task is an acceptable outcome. Budget exhaustion fell from 34.0% to 6.7%, and runs with an unsafe request after the evidence fell from 3.89% to 0.68%^[3].

The gain depends on the model. It was large for some and near zero for one open-weight coding model, Qwen3-Coder-30B, which rarely picked up the tool at all^[3].

For models like that, the paper added a rule-based monitor to the harness that spots the failure pattern and prompts the agent to review what is left, which is the backup when a model ignores its exit.

Does a way to stop work if you add it late?

Timing matters as much as the tool. Anthropic also replayed the incident this summer at the UK AI Security Institute, where Mythos 5 attacked a real software project during a hacking test.

In those replays, Mythos 5 "never once decided to stop its attack" in the turns it was tested on, and an abort option written into its notes partway through the run was mostly ignored^[4].

Anthropic tried three abort rules. The two general ones, stop if the environment looks misconfigured and stop if continuing risks real-world harm, had no effect at any point. Only the third, which named the exact act to avoid, worked, and only early: 46% of samples at the first point tested, 3% at the next, zero after that^[4].

Anthropic's text says the model did not abort in any cycle after the third; its chart shows a 3% rate in the fourth.

Either way, the side panel shows what an exit added late looks like from the inside: the model reads the rule, reasons that it must mean something else, and carries on. Anthropic also found it was very hard to move Mythos 5 off the attack "unless we offered some other, clearer path toward completing the challenge"^[4].

Newer models are better here. Replayed on the same transcript, Anthropic's later models began the attack between 0% and 13% of the time, against 83% for Mythos 5^[4]. Better training narrows the gap. It does not give you a design you can check.

What should AI agent guardrails include?

Keep the walls and the detection, and add an exit the agent can see from its first step: a stop tool, an instruction that stopping is a valid result, named triggers, and a handoff note. The walls stop known bad actions. The exit covers the cases nobody has built a wall for yet.

In practice, that is five changes to how an agent's task is set up:

• A stop tool from step one. A named tool, such as escalate to a person, available on every run, not added when something goes wrong.
• Stopping written into the task. One line in the instructions saying that not finishing is an acceptable outcome when the task cannot be done within scope.
• Named triggers. The task looks impossible within scope, the next step would leave scope, or the next resource is paid, gated or behind a login.
• A handoff note. What the agent tried, what it verified and what it needs. In ImpasseBench, 35.7% of escalation tickets left the agent's own progress and verification status only partly described^[3], which is the part the next person needs.
• A monitor as backup. For models that ignore the tool, something outside the run that watches for repeated failure and prompts the stop.

This is the same idea as the harness decides what reaches the world, applied to the case where the task cannot be done. It also keeps the human in the loop as part of the design rather than someone watching a dashboard: the agent brings the person in, with a note they can act on.

Anthropic's new tooling blocked every case it was tested against. That is the wall working. The question for any team running agents is what their agent does the next time it reaches a wall nobody built yet, and the answer should be that it stops and says so.