Anthropic’s Project Deal found an Opus advantage buyers barely noticed

A market built for the test

Before trading began, Claude interviewed each employee about what they wanted to buy and sell, their prices, and how they wanted their agent to negotiate. Those answers became individual system prompts. The agents then operated in Slack without asking their owners to approve purchases or weighing in during negotiations.

There were four runs: one real exchange and three additional runs for research. In runs A and D, every participant had an Opus 4.5 agent. In B and C, participants were randomly assigned either Opus 4.5 or the less capable Haiku 4.5. Employees could see runs A and B in Slack, but were not told which was real or how the runs differed until trading was over.

This differs from much earlier research on agent negotiations, which used hypothetical goods or artificial datasets. Project Deal involved real people, real belongings, and promised exchanges.

In the mixed-model runs, Opus agents did better on several objective measures. Their users completed an estimated 2.07 more deals than Haiku users, with p = 0.001. The estimated advantage rose to 2.11 deals after accounting for differences between runs, with p < 0.001.

Items listed by Opus agents were 6.63 percentage points more likely to sell, though that result was not statistically significant: p = 0.057. Comparing 44 items sold in both runs B and C, but by agents using different models, Opus sellers received an average of $3.64 more.

The researchers also examined 161 items sold at least twice across the four runs. For the same item, an Opus seller received an average of $2.68 more, while an Opus buyer paid $2.45 less. Both estimates were statistically significant. They came from 782 completed deals, but trades were not randomly assigned in this analysis, so Opus may have benefited in part from finding better opportunities.

The gap was meaningful against the price of the goods: the median item cost $12 and the average was $20.05. In deals between an Opus seller and a Haiku buyer, the average price was $24.18; when both agents used Opus, it was $18.63.

Yet participants rated the fairness of deals almost identically: 4.05 for Opus and 4.06 for Haiku on a scale where 4 was the midpoint. Of the 28 participants who had Haiku in one mixed run and Opus in the other, 17 preferred their Opus outcome and 11 preferred Haiku’s. That difference was not statistically significant.

The prompt mattered less than the model

Aggressive bargaining instructions did not produce a statistically significant increase in the chance of selling an item: the estimate was 5.2 percentage points, with p = 0.43. Aggressive sellers who did make a sale received about $6 more on average, but they had initially asked for about $26 more. After accounting for asking price, the estimated difference fell to about $0.95, with p = 0.275.

Aggressive buyers did not pay less: the estimated effect was $0.56, with p = 0.778. In this pilot, the model’s capability appeared to matter more than the owner’s preferred negotiating style.

The agents also made choices their owners might not have expected. One employee bought a second snowboard identical to one they already owned. Another asked Claude to buy itself a gift; it chose a pack of 19 table-tennis balls for $3, which the employee allowed because it cost less than $5. A separate negotiation arranged a real dog-walking outing between two employees.

The experiment also exposed a question the headline numbers cannot answer: what happens when people cannot tell that they are getting worse representation? Haiku users received less favorable terms, but their fairness ratings were virtually indistinguishable from Opus users’. I think that gap between measurable outcomes and perceived experience is the study’s most important warning. The researchers say the experiment was not designed to explain how such inequality might persist, and that more research is needed.

The market was also unusually gentle: participants were volunteers given $100, not companies acting to maximize their own returns. The authors point to possible risks beyond bargaining power, including attempts to manipulate an agent into revealing information or quietly steering it toward unwanted actions. No legal or political rules yet govern models making deals for people.

Source: anthropic.com