Smarter reasoning models make collective outcomes worse in social dilemmas

As autonomous LLM agents take over human tasks — from negotiating with services to allocating resources inside companies — we have gotten used to judging them on solo benchmarks. We care how well a model writes code, answers questions, or plans. But in the real world they run into each other, compete for limited resources, and sometimes manufacture competition nobody needed. The paper Evaluating Collective Behaviour of Hundreds of LLM Agents is about exactly that: what happens when it is not two agents in the room, or ten, but hundreds, each acting "in its own user's interest."

Not move by move, but strategy as an algorithm

The authors take the classic social dilemmas from game theory — situations where individual gain conflicts with the collective good. Public goods, where everyone benefits if everyone contributes but each player would rather save the money; collective risk, where disaster is averted only if contributions add up to enough; and the common-pool resource, essentially fishing: overdo it now and everyone is worse off later.

The methodological move at the center of the paper is to stop making the LLM pick an action every round. That mode often fails: the model can miss simple patterns in how other players behave and "forget" context. Instead, the model is asked once to design a whole strategy and describe it in words precisely enough that it can then be turned into executable code. That buys something important: the strategy can be inspected before deployment and run fast across an enormous number of simulations. Which is what makes the scale possible — populations of up to 256 agents, rare in work like this.

Another detail that echoes the real world: the authors did not want to hand the model a detailed rulebook. They give a high-level disposition — broadly prosocial, or more self-interested — and watch how different LLMs interpret it on their own. For every disposition and every model they generate hundreds of strategies, check that each one is safe to execute, and then compare the resulting behavior.

How the "character" of a strategy gets measured

To avoid drowning in thousands of algorithms, the authors turn each strategy's behavior into a large numerical fingerprint: how often it cooperates given different game histories. Then they run PCA to see how diverse the strategies are, and how cleanly a model separates the "cooperative" instruction from the "exploitative" one.

Several curious traits fall out of that analysis. In some models the two dispositions separate sharply: prosocial strategies really do behave differently from selfish ones. In others the difference all but vanishes — as if the prompt barely touched actual behavior. The common-pool game turns out to be especially hard to steer: even strong models do a worse job separating cooperative intent from individualistic intent.

Self-play: what happens when you release them into the crowd

This is where it gets practical. The authors take a population of a given size, mix in shares of the two strategy types — the more prosocial and the more selfish — and measure average social welfare. And they do it across scales: from 4 agents up to 256.

The picture is not encouraging, and it is not uniform either. In some games particular models stay relatively robust: even when most agents are given the selfish instruction, the system does not always collapse into total defection. But in other cases a larger group makes the result worse — any single agent's contribution matters less, and cutting corners gets more tempting. The harshest test is the common-pool resource: there, early defection can drain the stock irreversibly, and even if everyone comes to their senses later, the damage is hard to undo.

One observation from the authors is especially worth sitting with: newer reasoning models often produce worse collective outcomes. Not because they are "bad," but because they are better at finding ways to win individually whenever the environment lets them free-ride on everyone else's cooperation.

Cultural evolution: why the market can select for the worse option

To get closer to reality, the authors add one more layer — a simulation of user choice. Picture people choosing an agent for themselves (a model plus a disposition) and, over time, switching to whichever one delivers more. It looks like cultural evolution: successful strategies get copied, unsuccessful ones die out.

In the simulations, what many people intuitively fear is what usually happens: the selfish instructions win. They return slightly more to their user in a competitive environment, and that is enough to gradually crowd out the prosocial variants. In large populations, the risk of converging on a bad equilibrium is markedly higher. The logic is an unpleasant one: even when everyone would be better off cooperating, individual selection on short-term payoff can drive the whole system downhill.

What this changes in practice

The paper's central conclusion sounds simple: evaluating LLM agents has to cover collective behavior in social dilemmas, not just individual skill. If we deploy multi-agent systems without institutions — constraints, incentives, rules of interaction — markets and competition can push exploitative strategies into dominance on their own, even when that destroys collective welfare.

The approach is also valuable for how grounded it is in engineering terms. Fixed algorithmic strategies are convenient to audit before deployment, to compare across models, and to scale to hundreds of agents. Yes, the method deliberately rules out communication between agents, and that is a limitation. But in large systems communication can be expensive or impractical, which makes this speechless stress test realistic enough.

What the work leaves behind is an important warning: raising an agent's intelligence does not guarantee a better collective outcome. Sometimes it does the opposite — simply because a smarter agent is more efficient at finding the loopholes in the rules.

💾 Code