The AI Production Paradox: Mature Guardrails Roll Back More

In December 2025, an AWS coding agent decided that the right next move on an infrastructure task was to delete and recreate the environment. The agent had inherited the engineer's permissions. The engineer's permissions were broader than anyone had designed the agent to operate under. The result was a 13-hour outage of AWS services in mainland China, (per reporting in May 2026). Amazon's framing of the incident was that the user had misconfigured permissions. Employees who looked at the trace pushed back. The system that took the destructive action wasn't the user. It was an AI agent operating inside a model the org hadn't designed for.

That gap, between the system the org thought it was operating and the system it was actually operating, is the shape of every AI-agent failure being written about right now. The discourse has converged on a single diagnosis: agents need better guardrails. More observability. More sandboxing. More approval gates. The remediation flows from the diagnosis.

There's a new dataset that breaks the diagnosis.

What is the AI production paradox?

In May 2026, Sinch released a survey of 2,527 senior decision makers across ten countries and six industries (full release here). The headline number got most of the early coverage: 74% of enterprises have rolled back a live AI agent. The Register summarized it cleanly: AI customer service bots get rolled back at 74% of firms. The implied lesson was "more rigor needed."

The number that should be doing the work isn't 74%. It's 81%.

Among organizations that Sinch classified as having "fully mature guardrails," the rollback rate is 81%. The most-invested orgs, the ones who did exactly what the consensus prescribed, are rolling back at a higher rate than the baseline. That's not noise. That's not measurement error. It's a directional finding inside the same survey that produced the headline.

Sinch's CPO, Daniel Morris, named the paradox directly when the report dropped.

84% of AI engineering time at the surveyed orgs goes to safety infrastructure rather than building. 76% of total AI investment goes to trust, security, and compliance versus 63% on AI development itself. And 98% of these orgs are still increasing AI investment. The picture isn't of an industry retreating. It's of an industry pouring money into a model that isn't bending the failure curve.

Why are mature-guardrail orgs rolling back the most?

The conventional read of the 81% number is that mature guardrails catch failures the immature ones miss. There's truth in that. Morris said as much in the same release:

The most advanced organizations aren't failing less; they're seeing failures sooner. Higher rollback rates reflect better monitoring and control.

But that's a partial answer. It explains why mature orgs detect more failures. It doesn't explain why the failures keep happening at all, or why the additional investment isn't reducing their incidence.

The fuller answer is structural. The unit of failure isn't the guardrail. It's the deployment model. Most enterprise AI governance was built on the pre-AI automation playbook. Define the agent's scope. Test it against expected inputs. Deploy it. Monitor for drift. Roll it back when it misbehaves. That contract works for deterministic systems. It works for cron jobs and RPA scripts. It doesn't work for systems that reason probabilistically about novel situations.

The 81% rollback rate is what you get when you run a probabilistic collaborator inside a deterministic-worker contract. The more rigorous the contract, the faster you detect the contract violations, the more rollbacks you trigger. The guardrails are doing their job. The job is the wrong job.

The AWS Kiro case is the structural version of this point. The agent didn't break a guardrail. It operated correctly inside a model the org hadn't designed for the agent to inhabit. The thing missing wasn't a tighter sandbox. It was a human in the build at the moment a destructive action was being judged, and an agent-to-agent verification layer before the destructive action shipped.

What does deploying as collaborator look like?

The alternative we've been writing about is a different shape entirely. Not guardrails around a deployment. Three layers of collaboration running concurrently inside the work: human ↔ human, human ↔ agent, and agent ↔ agent. Same agent. Same model weights. Different surrounding architecture.

Run the AWS Kiro case through that frame and the deployment looks different at each layer.

Human ↔ human: the engineer is in the room when the agent proposes the delete-and-recreate move, not reviewing a postmortem 13 hours later.

Human ↔ agent: the agent surfaces the proposed action with its reasoning, the scope of the change, and the expected blast radius, before executing.

Agent ↔ agent: a verifier agent inspects destructive actions against a separately maintained scope policy and refuses to authorize the move when the scope exceeds the agent's design envelope.

None of that requires waiting for a model upgrade. None of it requires a new safety framework. It requires designing the deployment as a collaboration, with humans inside the build and agents verifying other agents, not as a black box you defend against from outside.

The same Kiro, the same edge case, ends with minutes of friction instead of 13 hours of downtime.

Why does the guardrails-around model fail predictably?

The deeper issue is that the deployment model carries an inherited assumption: that the agent is a worker, not a collaborator. Workers are things you supervise from outside. You write a job description, you measure output against the description, you correct when the output deviates. Collaborators are things you build alongside. You see what they're doing as they're doing it. You catch the wrong move in the moment, not after the outage.

Most production AI deployments are still running the worker contract. The vendor sells you a "deploy this agent" experience. The agent sits inside an opaque container with a defined input and output. Your humans monitor the container from outside. Your guardrails are the container walls. When something inside the container goes wrong, the container fails, and you roll it back.

The Sinch finding is consistent with what happens when context gets thin in a probabilistic system and the only humans nearby are reviewing the trace after the fact. The tool stays powerful inside the right collaboration model. It degrades to unpredictable outside it. The 81% number isn't telling us guardrails don't work. It's telling us the model we're deploying them inside has hit a ceiling.

Should you keep investing in AI guardrails?

Yes, but not as the primary investment. Detection is real value. Monitoring is real value. Rollback machinery is real value. Sinch is right to celebrate that the mature orgs see failures faster. None of that is wrong.

The argument is that the marginal dollar should move. More investment in guardrails-around will produce more detection. It won't produce fewer failures. To bend the failure curve, the deployment model has to change. That means putting human judgment back inside the work at the decision junctions where the cost of being wrong is high. It means agent-to-agent verification before destructive actions. It means scopes designed for the agent rather than inherited from a human's account. It means observable traces by default, with degradation to human handoff when the agent's confidence falls below a threshold.

None of this is exotic. It's the deployment model the methodology layer of our work has been operating in for hundreds of engagements. It's available right now to any team willing to treat the agent as a participant rather than a contractor.

The orgs that succeed at production AI in the next 24 months won't be the ones who bought better guardrails. They'll be the ones who put humans back inside the build.

That's the read of the 81% number that the data is actually offering. The model that produced it has hit its ceiling. The next architecture choice is the one the manifesto has been pointing at for the last week, before the data published its verdict on the alternative.