Anthropic Study: Domain Experts Unknowingly Adopt AI Agent Preferences
An Anthropic study reveals causal inference experts adopted AI agent preferences 86% of the time, believing the decisions were entirely their own.
A study conducted by Anthropic involving 50 domain experts in causal inference reveals that people frequently adopt the subtle preferences of AI agents without realizing it. Participants in the study analyzed observational data from a fictional agricultural cooperative to evaluate productivity gains from a voluntary advisory program. Although the experts retained final decision-making power, the choices made by an accompanying AI assistant heavily steered their analytical paths and ultimate conclusions.
How the experiment altered expert choices
To measure the agent's sway, researchers designed a multi-step task involving five separate decisions, each with two valid methodology options. Unknown to the participants, the AI agent's position for each option was determined entirely by a coin toss. The agent was restricted from lying or altering data, but it was programmed to present the strongest honest argument for its assigned side.
The results showed that experts embraced the AI's randomly assigned preference in 86 percent of the four core analytical decisions, such as picking statistical methods or defining productivity metrics. The agent's net influence, termed the "flip rate," swung choices by 62 to 82 percentage points across analytical tasks, compared to a 17-point shift on the final policy recommendation. Holding a prior view before the AI spoke provided minimal defense: participants with an opposing opinion were 53 percentage points more likely to change their mind to match the agent than those who initially agreed with it.
None of the potential protective factors tested by the authors—including years of analysis experience, test scores, self-rated expertise, education level, or general trust in AI—showed a statistically reliable link to resisting the agent's influence after adjusting for multiple tests.
Illusion of control and performance tradeoffs
Despite the heavy steering, participants maintained a strong belief that they were in full control. After completing the task, 52 percent felt the final policy recommendation was more their own than the AI's, while another 35 percent stated it was mostly or entirely theirs. However, 77 percent of participants incorporated at least one of the agent's arguments into their explanations—frequently using nearly identical wording—while only 15 percent referenced the AI at all.
This subtle deference coincided with clear efficiency gains. Working without an AI assistant resulted in serious methodological or coding errors in 55 percent of analyses (6 out of 11) and a median completion time of 90 minutes. With the AI agent present, serious flaws dropped to just 6 percent (3 out of 50), and average completion time fell to 65 minutes. However, because the small non-AI control group chose not to use the tool and had slightly less expertise, researchers noted the exact driver of this performance gap remains uncertain.
What it means for developers
For software engineers and system architects building agentic applications, these findings highlight crucial design considerations. When building complex agent workflows—such as autonomous code auditors, data analysis assistants, or AI-driven evaluation frameworks—agents that state a position first can unintentionally anchor human users.
When integrating LLM capabilities into development pipelines, developers can experiment with and evaluate multi-model behaviors cheaply using single-endpoint services like https://apixoai.online, which provides pay-per-token access to top models like Claude, GPT, and DeepSeek through one API key. To counter cognitive bias in agent design, developers may need to structure prompts so agents present balanced arguments for both sides or prompt human users for their input before the AI delivers a recommendation.
The Anthropic researchers warned that unexamined AI deference could lead to a "false consensus" in scientific research or compromise AI safety checks, where human reviewers are meant to act as independent evaluators for frontier model alignment. As the study authors noted, "There may be no single right division of labor between people and AI. Some decisions may be fine to hand over, and others worth keeping."
Wider impacts on human judgment
The findings align with broader research on how AI interaction alters human decision-making. A cited study of 3,132 participants showed that viewing AI-generated answers reduced the percentage of people willing to state "I don't know" from 44 percent to just 3 percent. Polished AI outputs also led to fewer fact-checks (down 3.7 percentage points) and less questioning of reasoning (down 3.1 points).
Similarly, a Polish observational study published in The Lancet Gastroenterology & Hepatology observed that after experienced doctors routinely used AI during colonoscopies, their rate of detecting precancerous lesions during exams without AI dropped from 28.4 to 22.4 percent. Conversely, other real-world data points to potential benefits, such as a Nigerian pilot study where students using an AI chatbot tutor twice weekly achieved learning gains over six weeks equivalent to nearly two additional years of education.
Source: Experts adopt AI agent's random opinions and think they're their own, Anthropic study finds — The Decoder. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
