Judge an Agentic OS by Where Your People's Hours Go

A research team at the University of Passau spent four weeks building software with up to nineteen AI coding agents and two people working in one shared room, and they released the full record.^[1]

One release from it carries most of the story. It went into production 9.9 hours after it was merged into the main codebase, and the agent responsible replied within minutes the whole way.

Before it started, that agent had predicted where the time would go, and it was right: "The time is in the gates, not the work." The paper traces where it went:

• The automated build-and-test run failed on two defects the local tests could not show.
• An unrelated change cancelled a full run, and nobody noticed for 45 minutes.
• A person's go for production stopped at an expired login until that person renewed it.

The agent could not read its own test results, so every verdict passed through a person.

That release is the question this autumn's agentic operating system pitches skip. Everyone is arguing about what the layer should contain or connect. Almost nobody is asking where the person's hours go once they are inside it.

What is an agentic operating system?

An agentic operating system is the layer where people and AI agents do work together: what an agent may touch, who it acts for, and how its work reaches a person. The phrase has two camps right now. One judges the layer by how well it contains agents, and the other by whether people and agents share a plan.

The two camps

The containment camp has the cleanest line. Microsoft built its new execution containers for agents on Windows around it: "An agent cannot be its own security authority."^[2] The policy lives outside the agent, enforced whatever the agent decides.

The collaboration camp is the one moving this week. Asana pitches "an operating system for human-agent teams," where people and agents work "from the same plan, with the same context, under the same governance."^[3] Perplexity launched Teammates on November 4, agents that "present results for review" and take work from colleagues in a Slack channel.^[4]

Both camps are describing something real, and we have argued before that when agents work together, the coordination layer is the story. Neither camp says what a person's day looks like inside that layer. Until last week there was no field data on it.

What did the first field study of a human-agent team find?

The Passau study, published October 31, followed up to nineteen coding agents and two people through 7,457 messages over four weeks as they built a distributed system. Its headline finding is that the agents were never the slow part. Requests waited on the human-run coordinator and on approvals, while the agents answered within minutes.^[1]

The wait grew with the team. As more agents joined, the coordinator answered requests less often, and the slowest of its response times stretched from half an hour to more than four hours. The authors name human attention and decision making as the critical resource.

That part is familiar by now; coding-agent teams have been saying attention is the constraint all autumn. What this study can show, and the commentary cannot, is what the attention was spent on.

Where did the person's hours go?

Mostly into checks, not decisions. A replay of 45 release chains found 148 hours spent waiting for the coordinator's checks, a mean of 3.3 hours per chain, against a deployment step that took a median of eight minutes. The operators say most waiting decisions were simple once someone looked.^[1]

The pattern shows up in smaller numbers too. Agents could not read their own build-and-test results and asked someone else to do it 60 times. The median wait was about half an hour, 13 waits ran past four hours, and the longest lasted 15.7 hours.

The authors put the conclusion plainly: "The bottleneck was therefore neither programming nor the difficulty of the decisions, but access to a human decision maker."

Read the ledger and the person's role comes into focus. They were not spending those hours on judgment calls; they were the instrument that read the result, which is a person reduced to a checker by design rather than by fatigue.

Why the room had no receipt

The deeper cause is that work was reported in sentences. The team invented a fixed vocabulary for partial results ("built, not deployed", "gates not run here") and typed it 802 times. Only 15 of 153 acceptance criteria were ever reported as met, partly because for a time no agent could report one.

Problems hid the same way. Thirty-eight of the 69 problems one coordinator logged were silent: no signal reached anyone until someone looked.

In one incident, a failed check after a deployment went to its owner as a for-your-information note instead of a request, so it started no work. It sat for 11.5 hours, and the repair then took 26 minutes. In the authors' words, "a claim about a module traveled further than the module."

Why isn't containment enough?

Containment is necessary, and the study backs it: no defense inside the conversation reliably stopped a planted backdoor. But containment governs what an agent may touch, not how a person's time is spent. A fully contained agent can still make a person the bottleneck by reporting work someone has to check by hand.

The study tested the collaboration camp's instinct too. Making the room fully transparent, so every participant could see every message, helped the team produce more working code, and the authors found no evidence that it reduced the backdoor.

The only reader that reliably caught the backdoor was a stronger model reading the record after the fact, which is the case for a record the agent cannot edit. So containment is the floor, though it says nothing about the person's hours.

How should you judge an agentic OS?

By how many of the person's hours go to confirming what an agent said versus deciding what happens next. The paper's own lesson points the same way: give agents the capabilities they otherwise wait for, such as reading their own build results, filter what reaches a person, and measure first response, completion and release separately.

That is a design target, and it changes what the room should do when an agent says it is finished. We run a system internally where a team of agents and the person who runs them work in one shared space, built around these rules:

• Each agent's authority lives in a policy-gated toolkit, not in its instructions.
• A fact counts only when a real tool returned it, so "it's shipped" shows as an unbacked claim until the build and the deploy confirm it.
• A request closes only when the person who asked acknowledges it.
• A review comes back approved or with changes requested, never as a paragraph.
• Spend runs against a daily cap per client.

It is early, and we have no numbers to publish. The point is where the design puts the person.

The difference from the Passau room

The Passau room had the same closure rule we use: "A goal is complete when the participant who opened it declares it so."^[1] It still cost the people 148 hours, because the check came from a person reading results the agent could not see.

The difference is whether the confirmation arrives in the room by itself or has to be fetched by someone. That gives three questions for anything calling itself an agentic OS, whether it is Windows, Asana, Perplexity or your own setup:

1. When an agent reports done, does the room show proof from the real system or a sentence from the agent?
2. When a request is waiting, can you see who it is waiting on?
3. How many of your hours last week went to reading results an agent could have been handed?

Those are the human-agent layer of the three layers of collaboration, and all three are measurable.

What the study can't tell us yet

It is one team, and the two people in it were the authors, building the system they were studying. One model agent coded the field data and no human checked it. The authors say the results may not transfer to larger organizations, and we read it as the first field data pointing a direction, not as proof.

They also expect the bottleneck to grow with more stakeholders, not shrink. The fixes the team reached for were organizational: more decision makers, delegated authority, and fewer messages competing for a person's attention.

Containment keeps the agents inside the room, and the layer worth building is the one that gives the person their hours back.