Is AI Making Us Dumber? Three Randomized Trials Say It Depends What You Ask It For
Three threads said the same thing inside 48 hours this weekend. A Berkeley study found that ten minutes with an AI assistant erodes your ability to persist at hard problems.[1] Piotr Sarnacki's essay about DHH's agent-written rewrites of Campfire hit the Hacker News front page under the title "I'm sorry, but you still have to think."[2]
Then Andrej Karpathy posted that the gap in understanding what these systems do is "widening" into a funnel, with "human review and comprehension starting to fall behind."[3] Fortune had run the economist's version the day before: David Autor, who co-authored a three-month trial with patent lawyers, called AI "a performance equalizer, but a skill-disequalizer."[4]
So the verdict that replaced "AI is the great equalizer" is "AI makes you dumber, unless you were already good." Both verdicts are about the tool. The trials are about the person, and in the trials' own breakdowns, the variable that decides who keeps their skills is what they asked the machine to do.
Was AI ever the great equalizer?
On the work people hand in, yes, and the evidence has not gone anywhere. In every large trial since 2023, AI assistance lifted the floor: novices gained the most, output quality rose, and the gains held for months. The equalizer story was true for assisted output. The new trials asked a different question.
Brynjolfsson, Li and Raymond followed 5,179 customer-support agents: productivity up 14 percent on average and 34 percent for novice and low-skilled workers, with minimal impact on the experienced ones.[5]
Dell'Acqua and colleagues gave 758 BCG consultants the tool: 12.2 percent more tasks, 25.1 percent faster, higher quality on tasks inside the tool's reach, and 19 percentage points less likely to be correct on the one task chosen to sit outside it.[6]
Two more this year
Autor's lawyers produced better drafts with the assistant, up 0.38 standard deviations at 90 days, with the larger gains among juniors.[8] An OpenAI and Bocconi trial with 1,053 first-year students found that ChatGPT raised expert evaluations of novices' business solutions, while a course in causal reasoning changed how they thought and did not move the grades.[7]
OpenAI staff are among the authors of that one, so read it as a vendor's result. The direction matches everything else, though. The tool lifts the floor on the work you hand in, and nobody serious disputes it.
What changed in the 2026 trials?
The 2026 trials measured the person after the tool was taken away. Autor's lawyers sat an AI-free redlining test after three months. The Berkeley and COLM team gave people a chatbot for twelve problems and removed it for the last three. Both designs separate what the tool did for the work from what it did to the worker.
Start with the lawyers. Autor, Rodchenko and five Google co-authors randomized 133 patent lawyers at eleven US firms, about two-thirds to an unreleased Google Labs drafting assistant, and had blinded attorneys at an independent firm score the work.[8]
On the unassisted redline, treated lawyers beat controls by 0.32 standard deviations. Seniors, meaning seven or more years in practice, beat them by 0.45. Juniors showed no average gain.
The caveats belong in the same breath.
It is a working paper, not yet peer reviewed. Only 91 of the 133 lawyers finished the AI-free test. Fifteen of the 91 markups were flagged as possibly AI-assisted despite the ban, about as often among controls as treated, and with them excluded the senior advantage is 0.39 standard deviations at a looser statistical threshold.[4]
The firms declined a baseline skills test. Google built the tool and paid the study's direct costs, and six of the seven authors work there. Fortune also notes the study sorted by years of practice, not age.
Ten minutes and a stopwatch
The persistence study did the same thing with 1,222 people and far less time. Liu, Christian, Dumbalska, Bakker and Dubey ran three pre-registered trials on Prolific.[10] In the first, 307 people after screening got fifteen fraction problems, with ChatGPT in a sidebar for twelve and nothing for the last three.
On the three unassisted problems the AI group's solve rate was 0.57 against 0.73 for people who never had it, and their skip rate was 0.20 against 0.11.
A replication with 667 people and a pretest gave 0.71 against 0.77. A third trial with 168 people on SAT reading, with GPT-5 in the sidebar, gave 0.76 against 0.89. Ten to fifteen minutes of exposure, tested immediately, with an assistant built to be maximally helpful.
"You almost couldn't ask for a clearer story," Brian Christian told Berkeley's research office.[1]
Who actually lost the skill?
The same paper answers it, and the headlines left the answer out. In the replication, people were asked how they had used the assistant, and the answers were checked against the chat logs. Sixty-one percent said mainly to get answers, 27 percent for hints or clarifications, and 12 percent did not use it.[10]
The groups had entered the study with no measurable difference in skill or motivation. On the unassisted test, the answer-getters solved 0.65 against 0.77 for the control group and skipped 0.13 against 0.07. The hint-users solved 0.76 and skipped 0.05. They were indistinguishable from people who never had the tool.
The authors are careful about this: the split is cross-sectional, not randomized, so it does not carry the causal weight of the headline result. It is also the whole story. The loss did not come from having the tool. It came from what the majority asked the tool for.
Thus, when people prompted AI to solve tasks for them, they were less likely to persist with tasks compared to people who didn't use AI or people who used AI to aid understanding.
The lawyers show the same shape from the other side
Autor's juniors did not fall. They split. Their scores on the unassisted redline spread out: more very low scores, fewer mediocre ones, more good ones, and no more excellent ones.[9]
The qualitative record explains the spread. Juniors in both arms tended to find a serious flaw and describe it in a comment without fixing it, a posture the authors call diagnose-without-execute. It was as common among the control juniors, so the tool neither caused it nor cured it.
The seniors who improved did something specific with the output. They spent longer on the redline, rebuilt the claims from scratch, stripped language that could narrow legal rights, tied edits to doctrine, and in interviews described treating the assistant's draft as a "logic auditor."[9]
One group asked the tool for the draft. The other asked it to check theirs.
January's Anthropic trial had the split too: 52 engineers, an AI group at 50 percent on the comprehension quiz against 67 percent for hand-coders, and a spread of more than 25 points inside the AI group by how they used the model.[11]
The engineers who asked it to explain scored at the hand-coding level. We wrote about that within-group gap in May, from one trial. There are now three.
Why is "it depends what you ask it for" the honest read?
Because every 2026 trial, when it looked inside the AI group, sorted people by what they asked the machine to do. The between-group headline is what that sorting looks like from outside.
Years of practice made the better ask more likely, which is why the headline lands on seniority. But the juniors who fixed what they found moved up, the hint-users held level, and the inquiry-mode engineers matched controls.
Our data suggest that it's a performance equalizer, but a skill-disequalizer, in that only practitioners who already had foundational mental models were able to level up their underlying skill sets.
"Skill-disequalizer" is the first-order description, and it is fair. The second-order one is that a mental model is one source of direction, and the trials keep finding that direction, not tenure, is what the tool amplifies.
That is the manifesto line with a sample size. AI amplifies your direction, right or wrong, and the result is built from your expertise before it can go anywhere beyond it.
What do the professionals who use it all day already know?
They behave as if they had read the trials. Stack Overflow's 2026 survey, fielded June 23 to August 5 with 30,903 valid responses, found 66 percent of developers using coding assistants or agents and 73 percent of those using them daily.[12]
Yet 48 percent trust AI only when they can easily validate the answer, 16 percent extend trust to unimportant tasks, and 6.6 percent trust it for important decisions.[13] Nearly four in five rate source attribution as important or very important.
Look at what they hand over, by the share of developers who use AI for each task:
- Code in a familiar area: 69 percent
- Debugging and refactoring: 64 percent
- Writing tests: 58 percent
- Communication and design, where there is no checkable answer: 35 and 49 percent on usefulness
Developers give the tool the work they can check and keep the work they cannot. That is a direction decision made at scale, and 17 percent of the people holding back name losing skills as the reason.
The essay and the funnel, re-aimed
Sarnacki's essay is right and slightly under-aimed. DHH's three rewrites of Campfire differ on the CSRF token, on Redis versus in-process queues, and on blocking calls inside async tasks. None of that was in the prompt, so in his words "many decisions are a coin flip."[2]
His conclusion is that you still have to read the code. The trials point at the earlier move: make the decisions the prompt left open before generation, so the software is built on your choices instead of the AI's, and then read the code to find the ones you missed.
Karpathy's own account of the funnel helps here. Capability, he writes, "peaks in domains that are digital, verifiable and economically valuable."[3] The tool is strongest exactly where you can check it, and developers already route their work that way.
What should a founder or a team lead do with this?
Treat the trials as a recipe rather than a warning. Each one found a group that kept its skills while using the same tool as the group that did not, and the difference was a habit, not a ban. Three habits cover it.
- Ask for the hint before the answer. The cost in the data was zero: hint-users finished the same twelve problems with the same sidebar. The benefit was the entire gap.
- Decide the constraints first. Write down the choices the prompt would otherwise leave to a coin flip, so the model is auditing your decisions instead of making them. The seniors who improved described exactly this posture.
- Take the tool away on purpose. Autor's redline test is something a team can run on itself once a quarter. If the unassisted work is not improving, the assisted work is masking it.
For juniors, the problem the trial found predates the tool. Diagnose-without-execute was as common among controls, and it is fixed by practice fixing, not by a ban on the assistant.
The persistence authors add a warning that belongs here. Socratic modes and usage limits may only be band-aids while the systems are optimized for short-term helpfulness, so the habit has to be yours.[10]
"Is AI making us dumber" has the wrong subject.
The trials measured people, and what each person did with the answer decided what they kept. Built from your expertise, beyond what expertise alone could reach: the first half turned out to be a precondition, and the second half turned out to depend on whether you asked a question or placed an order.
References
- ^1.UC Berkeley Research, “Using AI for Just 10 Minutes Erodes Your Ability to Persist at Hard Things” (October 9, 2026)
- ^
- ^3.Andrej Karpathy, X, “The gap in shared understanding of LLM capability is widening” (October 11, 2026)
- ^
- ^
- ^
- ^7.Asirvatham et al., OpenAI and Bocconi, “Training novices to think, or giving them LLMs? Evidence from an RCT” (August 2026)
- ^8.Autor et al., NBER Working Paper 35720, “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting” (September 2026)
- ^9.Autor and Rodchenko, Google Research, “Does better work always mean better workers?” (October 7, 2026)
- ^10.Liu, Christian, Dumbalska, Bakker and Dubey, COLM 2026, “AI Assistance Reduces Persistence and Hurts Independent Performance” (October 3, 2026)
- ^11.Shen and Tamkin, Anthropic, “How AI Assistance Impacts the Formation of Coding Skills” (January 29, 2026)
- ^
- ^13.Stack Overflow, “Stack Overflow Annual Developer Survey Finds AI Trust Is Conditional” (October 6, 2026)
Frequently asked
Is AI making us dumber or smarter?›The randomized trials published this year say it depends on how you use it, not whether you use it.
Is there evidence that AI tools reduce critical thinking skills?›Yes, with a specific shape. The COLM 2026 persistence study found lower unassisted solve rates and higher skip rates across three experiments after brief AI use.
Does AI help experts or beginners more?›On the work people hand in, beginners gain the most, and that result has held since the 2023 call-center study.
Can AI tools make people less likely to verify information independently?›Professional developers already behave as if they knew the risk.
How can you use AI without losing your skills?›The trials point at three habits. Ask for the hint or the explanation before the answer, because hint-users matched the no-AI group in the persistence study and inquiry-mode engineers matched controls in Anthropic's study.
Is AI an equalizer for junior employees?›For the work they hand in, largely yes. The patent trial raised junior drafting quality and an OpenAI and Bocconi trial with 1,053 first-year students found ChatGPT raised expert evaluations of novices' solutions.
Let’s build it together.
We turn clever prototypes into production systems people can rely on. If you’re building with agents and want a hand making it real, leave your email and we’ll be in touch.