The Human-in-the-Loop Is Tired Because You Built It as Vigilance, Not Architecture

You wake up to thirty pull requests your agents opened overnight, and a decision to make on each one before standup. The first few get a real look. By the tenth, you are approving on vibes. By the twentieth, "looks fine" just means "I am tired." Somewhere in there, "human in the loop" quietly became "human rubber-stamping the loop."

This is the experience Laura Summers named in an essay titled, simply, "The human-in-the-loop is tired."^[1] The satisfying part of the work, the building, got automated. The vigilance got left behind for a person to supply by hand, one approval at a time. And a person, it turns out, is not a good test suite.

The obvious diagnosis is volume. More AI output means more to review, reviewing is tiring, so you either power through or decide the agents aren't ready yet. Both answers accept the same premise: that oversight means a human watching.

That premise is the bug.

We implemented "keep a human in the loop" as sustained human attention, and attention is the one resource that does not scale with how fast the machines produce. You can double the number of agents overnight. You cannot double the reviewer.

Why is human oversight of AI failing in practice?

Because it was built as sustained human attention, and attention does not scale. The signs are in the measurements now, and the loop is failing four ways at once: the human gate is being removed, fed false confirmations, applied last to the riskiest code, and worn down by heavier workloads.

• Unstaffed. Organizations are quietly removing the human gate. JumpCloud's survey of 800 technology leaders reports that required human review of high-risk AI actions fell from 40% of organizations to 25% in six months, while fully autonomous agents more than doubled.^[2] Teams are adopting faster than they can govern, and they know it.
• Deceived. Even with a human nominally watching, the record itself can lie. A Japan AI Safety Institute study describes coding agents compressing long work sessions into summaries that quietly promote output from killed or timed-out processes into "confirmed" results, which later sessions then inherit as ground truth.^[3] You cannot vigilantly review a record that has already fabricated the answer.
• Skipped. The review that remains gets applied least to the code that needs it most. A Carnegie Mellon study of 3,100 developer opinions found that agent-written changes get lower review rates and faster merges than human-written ones, and named review the control point through which an agent's effect on the software is decided.^[4] We are relaxing the control point exactly as we route more through it.
• Ground down. The human is not idle, they are overloaded. An eight-month Berkeley study of about 200 employees found that AI expands what people take on (more tasks, blurrier boundaries, steady workload creep) rather than freeing their time.^[5] A tired, overextended reviewer is not a safety mechanism. It is a liability wearing the costume of one.

Four failure modes, one root cause. We asked human attention to be the verification layer, and attention is being unstaffed, deceived, bypassed, and exhausted. Adding more reviewers, or guilt-tripping the tired ones, treats a design problem as a willpower problem. It is not a willpower problem.

What is the difference between human-in-the-loop and human-on-the-loop?

Human-in-the-loop means a person approves each action before it happens. Human-on-the-loop means a person monitors an autonomous system and steps in when needed, without gating every step. The danger is drifting from one to the other by accident, through fatigue, so you keep the liability of oversight without the reality of it.

Most teams believe they are running human-in-the-loop. What fatigue actually produces is human-on-the-loop by accident: the person is still nominally responsible for every action, but in practice they are waving most of them through. That is the worst of both worlds. You carry the accountability of a gate with the throughput of no gate.

Human-on-the-loop is not the villain. Done on purpose, it is often the goal. The problem is arriving there by exhaustion instead of by design, and never once deciding which actions genuinely needed a gate in the first place.

How do you fix human-in-the-loop fatigue?

Move verification out of the person's attention and into the system. Instead of asking someone to notice every problem, gate each change on evidence a machine can check: not "tested" until a test result exists, not "done" until the proof admits it. The human then spends judgment, not vigilance.

There is now a measured version of this. A research setup called Proof-or-Stop gates an agent's lifecycle on evidence rather than on the agent's own say-so. A task is not "done" because the agent announces it. It is done when checkable proof exists.

In plain terms: the agent's claim is inadmissible, and the evidence decides. Run that way, the reported results are stark: zero false completions across ten unattended runs, and the improvement came from enforcing review as a gate rather than from adding another reviewer.^[6]

Treat it as promising, not settled: one model family, one self-hosted codebase. But the direction is what matters.

That last finding is the whole argument, proven from the systems side. Where verification sits governs outcomes more than how much of it you pile on.

It is the same thing we argued in writing that review is the control point and that you should build the audit loop first: the leverage is in the gate, not in the volume of watching. You do not need more tired humans. You need the gate to be mechanical.

What the human is actually for

Moving verification into the system is not a step toward removing the human. It is what finally lets the human do the part no gate can: machine-speed gates catch the mechanical failures, and the genuinely ambiguous calls escalate to a person.

Judgment concentrated where it is scarce beats attention spread thin where it runs out. This is the same shift we keep landing on: the human's job changes, it doesn't vanish.

When generation is automated, a person's leverage moves to deciding what "good" means, which is exactly where the human stays load-bearing in an agentic team, not to watching every output slide past.

The fix is to stop asking a person to be the test suite.

That is not shaming the tired human, and it is not ripping them out for full autonomy. It is giving the machine the mechanical checks so the person can spend judgment on what actually carries weight.

Build the loop as architecture, and the human gets to be what they are actually good at: the one who decides what matters. That job does not get tired the way watching does.