Product Requirements for AI Coding Agents: Decide Before the Agent Starts

Sam Altman told Axios on September 7 that he wants to fund a kind of founder Silicon Valley used to laugh out of the room: "people who just really deeply understand their users and can't code at all."^[1] He called it a big turnaround, and he has a name for it.

In May, on stage with Patrick Collison at Stripe Sessions, he called this moment the revenge of the idea guys^[2].

I think he is right, and I think the phrase hides a job. The idea guy did not get the coder for free. He inherited the coder's hardest task, and a paper published six days ago puts a price on doing it late.

What did Altman actually say?

Altman said technical talent was the most important ingredient on a founding team, that it still matters, and that he now wants to fund founders who deeply understand their users and cannot code. The prevailing read is that the entrance exam is gone. The critical read is that it ignores the labor behind the tools.

Both readings skip the qualification he put in the sentence. "Deeply understand their users" describes someone who can make decisions about a product, and the funding line is his bet that those decisions are now the scarce input.

We argued in June that code was the slow part and the constraint moved up to ideas and judgment. Altman is saying the same thing with a checkbook.

What did the coder do besides code?

The coder turned an idea into decisions: what the product does first, what it refuses to do, which edge case matters, what to leave out. Those decisions do not disappear when an agent writes the code. They land on the person with the idea, who now has to make them before the agent starts.

Fred Brooks wrote the sentence for this in 1986: "The hardest single part of building a software system is deciding precisely what to build."^[4] He was explaining why no tool would make software ten times easier to build, and his reason was that the difficulty lives in the decisions rather than the typing.

Forty years later the typing is close to free, and the sentence reads like a job description. That founder used to hand the decisions to an engineer along with the idea, and the engineer made most of them without asking. An agent makes them too, faster, and with no idea which ones you would have made differently.

What does deciding late cost when an agent is building?

About twice the rework. In 3,553 real coding-agent sessions, a requirement that arrived after the agent had started was followed by 1.96 times the deletion of code the agent had already written, compared with matched edits that were not requirements. One session in five contained such an arrival, and more than half of them landed after the midpoint.

The study, from Karlsruhe Institute of Technology, Waseda and Adelaide, mines the SWE-chat corpus of real agent sessions^[3].

The 95% interval on that ratio runs from 1.31 to 2.82, which is wide, and it sits entirely above one. In lines, a late requirement was followed by about 28 more deleted lines on average, and the burden showed no detectable decline as sessions went on.

The authors' scoping is careful, and I will carry it. The sample is what they could reconstruct, matching cannot remove every confound about where in a task an edit falls, deletion is a proxy for invalidated work, and they describe the result as a temporal association rather than a demonstrated cause.

The timing is the part a founder should sit with. Between 18 and 22% of clean-start sessions carried at least one late-arriving requirement, and 54% of those arrivals came after the session's midpoint, which is when the most agent-written code is sitting there to be thrown away.

Can you just tell the agent to stay flexible?

No, or at least the measured version of that idea did nothing. In a controlled experiment from the same paper, telling the agent up front that a requirement would arrive later changed how much earlier code got overwritten by 0.16 lines, with an interval from minus 0.56 to plus 1.06. The obvious workaround has no detected effect.

The same experiment checked what a late reveal does to the shape of the work. Delaying the requirement moved the implementation into the round after the reveal, about nine more lines of churn there, which the authors treat as a check that the manipulation worked rather than as extra rework.

The point that survives both arms is simple. You cannot caveat your way out of a decision you have not made.

The token bill points the same way. A Stanford study of 2,700 runs on one model found that reducing a full task specification to a bare user story raised token spend by 29.7%, with the sensitivity ranging from 13% to 115% across tasks^[5]. Five tasks and one model, so carry it lightly. Vague is billed.

Why do requirements arrive late in the first place?

Because people cannot express a constraint until part of the system exists to react to. That is a fact about how humans discover what they want, not a failure of preparation, and it is why more than half of late requirements arrive after the midpoint.

The fix is a document that captures what you know, plus a build small enough to surface what you don't.

Two more results say what happens to the requirements you never write down. Microsoft's "Building to the Test" had two production Copilot CLI agents re-implement a data table as a reusable library under a hidden oracle of 222 tests, across 18 runs^[6].

Without the oracle the library was present but unfinished. With the oracle in the loop the score reached near-perfect, "but from a demo holding the tested behavior directly, the library left dead or absent."

The agent, in the authors' words, "does not, on its own, validate what it ships as a user would."

SWE-Gate, published September 3, found that of 644 agent repairs that passed their functional tests, 221 violated review constraints that real maintainers had imposed on the same kind of change^[7]. The requirements you write become the acceptance criteria. The ones you do not write get optimized away.

That is the practice behind two of the eight moves we published in June: name the things before the screens, and ship the smallest real thing early. The early build exists so the constraint you cannot articulate yet shows up while it is still cheap.

What should a product requirements document contain for an AI agent?

The decisions the agent cannot discover on its own: who the users are, what the product must not do, the constraints your market imposes, and the acceptance criteria you will actually check. An assistant can draft the format in seconds. It cannot supply the constraint you have not discovered yet.

People are searching for a product requirements document for AI, and the honest answer is that the template was never the hard part. The content the study says is expensive to discover mid-build is the content only the founder has: the users, the market, the things you refuse to build. The practical version fits in five moves.

• Write down who the users are and what the product must not do before the first build.
• Decide what you can decide, and do not ask the agent to keep it flexible. The measured effect of that request is zero.
• Write the acceptance criteria you will check yourself, because agents build to what you check.
• Build the smallest real thing early, on purpose, to surface the constraint you cannot articulate yet. The study says it tends to arrive after the midpoint.
• Budget rework at twice the normal rate for anything you leave undecided.

Where does the founder still need a partner?

At the point where reacting to a build requires reading it. Deciding what to build is the founder's job because only the founder knows the users, but turning a reaction into a decision the agent can execute, and checking the result as a user would, is a second skill. It is the one solo idea guys run out of.

Solo creation with AI has a ceiling. One person and one tool is the old paradigm with a new interface. Our clients are co-creators, not a source of requirements, and that model exists for exactly this seam: the domain expert supplies the understanding, someone turns it into decisions before the agent starts, and someone checks what came back.

When you find a person who deeply understands the users, put them in the room before the agent starts.

Altman is right that the coder is optional now. The product still needs someone to decide what it is, and the data says deciding late costs about twice as much as deciding early, with no credit for telling the agent you might change your mind.

The idea guy got his revenge, and the job that came with it is the requirements.