Experts defer to AI in decision-making, Anthropic study finds

Professional analysts sided with the AI assistant's preference in 86 percent of instances, findings from a recent study by Anthropic.

The research, conducted by the creators of Claude, involved 50 professional analysts advising a fictional farming cooperative. Their task was to assess the effectiveness of an advisory program for the co-op's members and decide if expansion was warranted.

Before finalizing a policy recommendation, participants were required to select four choices from two defensible options. Unbeknownst to the analysts, Anthropic programmed each AI assistant to favor a specific outcome to gauge influence on the human decision-making process.

The study revealed that this influence was significant; analysts chose options aligning with the AI's preference 86 percent of the time, far exceeding the 50/50 baseline expected without bias. Furthermore, when the agent disagreed with their initial leanings, participants shifted their decision 60 percent of the time.

Although 77 percent of analysts cited their AI assistant's arguments when explaining their choices, only 15 percent acknowledged AI as the origin of their own thought process, according to Anthropic.

Anthropic acknowledged limitations in the study, noting it involved only one task and utilized AI assistants deliberately programmed with a preference. Researchers Amaç Herdagdelen, Hailey Joren, and Alex Tamkin commented that the human inclination to yield to AI might be "hard to identify, even for the experts themselves."

These findings suggest that AI's inherent "priors" could influence major policy choices, even those concerning AI safety. The researchers advised that for human judgment to remain a vital check as AI integration deepens, oversight must examine *how* decisions are reached.

They further wrote that "Surfacing key decision points as the work proceeds and helping experts articulate the assumptions and values they would otherwise leave unstated" is necessary, suggesting a process allowing experts to uncover their own preferences could be as empowering as the final decision.