Analysts adopt the choices their AI agent was secretly told to favour

Anthropic recruited 61 analysts with experience in causal inference to advise a fictional farming cooperative on whether to expand an advisory programme, using Claude Code with Claude Opus 4.8. The task left four choices open, each with two reasonable options: the productivity measure, the comparison method, which farms the estimate covers and how farms are weighted. For each analyst and decision, a coin flip set which option the agent would favour, through hidden instructions that also told it never to mislead and to change its view if the evidence demanded it.

Among the 50 who used the agent, choices matched its lean 86% of the time, between 81% and 92% depending on the decision. When an analyst had stated a preference before the agent spoke, they switched away from it 60% of the time if the agent disagreed and 6% if it agreed, though that estimate rests on 38 cases. Afterwards 77% repeated the agent’s arguments, sometimes word for word, and 15% credited it. A model using only the four leans predicted 82% of final recommendations.

The 11 analysts who chose to work without the agent made serious methodological errors in six cases, against three of the 50 who used it, but they were less experienced on average. The authors note that they strengthened the agent’s preferences on purpose and studied one task with one model.