Anthropic study finds experts adopting AI agents’ random positions

Anthropic’s experiment found that experts often adopted an AI agent’s position without recognising how much it had shaped their judgment. Fifty specialists in causal inference analysed a fictional farming cooperative’s data and advised its board on whether to expand or end a voluntary farmer-consulting program. Unknown to them, the agent’s preference at each stage was chosen by a coin flip. It could not hide data or lie; it only had to make an honest, persuasive case for its assigned position.

Five decisions, two valid options for each, and an agent whose preference was assigned by coin toss. That's how the study measured the agent's influence. | Image: Anthropic

Source: the-decoder.com

Where the agent changed the analysis

Across four analytical decisions, participants followed the agent’s preference 86% of the time. The choices included how to measure productivity and which statistical method to use. Depending on the task, the researchers estimated the agent’s net influence—the “position-shift rate”—at 62% to 82%.

An expert’s initial view offered little protection. When the agent argued against a participant’s stated preference, the chance that the participant changed their decision rose by 53 percentage points compared with when the agent agreed.

The researchers found no statistically reliable relationship between participants’ results and their analytical experience, self-rated expertise, education, test scores or trust in AI after adjusting for multiple comparisons. Age stood out before that adjustment, but a sample of 50 gave the study limited power to detect such effects.

The share of participants who chose option A, based on whether their agent favored A (green) or B (purple). The rest chose B. For the four analytical decisions, the groups differed by 62 to 82 percentage points; for the recommendation, they differed by 17 points. | Image: Anthropic

Source: the-decoder.com

A human recommendation can still reflect an agent

The agent’s assigned position had only a slight, statistically insignificant effect on the final recommendation to the cooperative’s board. Some experts rejected its advice when it conflicted with their own assessment of the productivity gains.

But the final vote is a poor test of whether people retained control. The four earlier analytical choices strongly influenced participants’ estimates of productivity gains, which in turn shaped their recommendation.

Afterward, 52% said the recommendation felt more like their own than the agent’s; another 35% considered it mostly or entirely their own. Participants correctly recalled their choices in 85% of decisions. Yet 77%—37 of 48 people—used at least one of the agent’s arguments in their explanations, sometimes almost verbatim. Only 15% mentioned AI at all.

In 77% of exchanges, the agent stated its position first. It usually argued for its preferred option without explaining why a qualified expert might choose the alternative, though it did describe how that alternative would affect the result.

Working with AI was also associated with fewer errors and less time. Serious methodological or coding errors appeared in 55% of analyses without an agent—6 of 11—and the median work time was 90 minutes. With AI, serious errors appeared in 6% of analyses—3 of 50—and participants finished in 65 minutes. The comparison is limited: the no-AI group was small, had opted out of using the tool and was somewhat less qualified. The researchers therefore cannot isolate how much of the difference was caused by AI.

The tension is not simply that an agent can sway a final decision. It can shape the analysis that makes the decision seem like the expert’s own. I think that is the more consequential result: a human sign-off may offer little reassurance if the reasoning leading to it has already been steered.

The safeguards the study points toward

The researchers warn that similar defaults across widely used models could create a false consensus in research. They also flag a risk for AI safety: if experts rely on agents to assess advanced models, an agent’s unnoticed biases could shape the very research meant to uncover failures of alignment.

Their proposed responses include measuring how often people follow agents in real work, asking experts for their views before the agent states its own, and having agents present the strongest arguments on both sides. Institutions should also decide which choices require independent human judgment and provide the time and resources to make that judgment possible.

The paper does not claim there is one correct division of work between people and AI. Some decisions may be suitable for delegation; others may need to stay with a human. The challenge is making that division deliberate rather than letting an agent’s default position set it by accident.

Some explanations nearly matched the agent's wording. Participants reused its phrasing in their follow-up responses. | Image: Anthropic

Source: the-decoder.com

Evidence beyond this experiment

Other recent work points to related risks, though the findings are not uniform. In a study of 3,132 people on AI-induced overconfidence, seeing AI answers nearly eliminated people’s willingness to abstain: in one part of the study, the share willing to say “I don’t know” fell from 44% to 3%.

Presentation also mattered. When AI returned a polished result, such as code or a document, users were 3.7 percentage points less likely to check facts, 3.1 points less likely to question the reasoning and 5.2 points more likely to miss missing context. Users who refined results step by step questioned the AI’s reasoning 5.6 times more often than those who did not repeat that process.

A Polish observational study of colonoscopies published in The Lancet Gastroenterology & Hepatology found that after experienced doctors began regularly using an AI system, the detection rate for precancerous lesions in examinations without AI fell from 28.4% to 22.4%. A Swiss Business School review also found a strong negative association between AI-tool use and critical thinking, particularly among people aged 17 to 25. Higher education appeared to offer some protection.

These results come from early studies and individual snapshots, and other evidence points the other way. In a six-week pilot in Nigeria, students who used a chatbot as a virtual tutor twice a week made learning gains comparable to nearly two additional years of schooling. The broader question is not whether AI helps or harms in general, but whether its influence is visible enough for people to choose when to rely on it.