Reasoning models get better by arguing with themselves inside one trace

We tend to assume reasoning models are stronger simply because they write longer chains of thought and burn more compute before answering. In Reasoning Models Generate Societies of Thought the authors offer a more interesting explanation: these models don't just think for longer, they start thinking differently — as though a small assembly of voices had formed inside them, putting questions to each other, arguing, changing position and then settling on a shared answer. That is what they call a society of thought.

The problem the paper starts from is easy to state: LLMs can solve hard problems, but why some versions (DeepSeek-R1, QwQ-32B and others) reason markedly more reliably than ordinary instruction-tuned models is still unclear. If the secret were only the length of the chain of thought, printing more text would be enough — and in practice that doesn't always buy anything.

Reasoning as an internal conversation

The authors take 8,262 problems from the popular suites (BigBench Hard, GPQA, MATH Hard, MMLU-Pro and others) and look not only at final accuracy but at the reasoning traces themselves. Here is where it gets non-obvious: the traces are analyzed as dialogue. A dedicated annotation pass (an LLM judge, checked for agreement with humans and with other models) hunts for four conversational behaviors: asking and answering questions, shifting perspective, conflict of views, and reconciliation.

And here is what shows up: these patterns occur far more often in reasoning models than in base and instruction-tuned versions — even controlling for the length of the reasoning. The point is that the structure of the text comes to look like a team trading turns: one voice proposes a move, another doubts it, a third checks it, and then the answer is put together.

The authors go further and add a layer of social psychology: they classify socio-emotional roles by Bales's method — from support and agreement to tension and disagreement. Again the reasoning models turn out to be more social: their mix of roles is more balanced — not only friendly agreement, but ordinary working conflict, which in human groups often helps catch mistakes.

Is this more than style?

One of the paper's strong moments is the attempt to show causation. The authors use sparse autoencoders (SAEs) to pull sparse features out of the model's activation space. Among them they find feature 30939 — a marker that fires often at points resembling a change of speaker or a sudden realization in dialogue (something like “Oh!”).

Then they steer: at every generation step the model is nudged slightly toward that feature. On Countdown (multi-step arithmetic), accuracy at steering +10 jumps from 27.1% to 54.8%, while at steering −10 it falls to 23.8%. The conversational behaviors rise along with it: more question-and-answer, more clashes of perspective, more reconciliation. It looks as though switching on dialogue mode genuinely helps the model search, re-check and backtrack, rather than merely dressing up the text.

Different personas as a working tool

The other line of evidence is the search for implicit perspectives. An LLM judge tries to pick out how many voices sound within a single trace, and assigns each of them approximate personality traits (via BFI-10) and a type of expertise. Reasoning models, it turns out, show more variety: as if one inner voice were the riskier and more creative one, another more critical, a third pulling the problem toward its own expertise. The authors measure the diversity of personalities and of expertises separately, and show it is statistically higher in reasoning models.

Importantly, none of this is served up as mysticism about consciousness. It reads instead as a useful computational arrangement: to cover the solution space, a model does well to generate competing hypotheses, set them against each other, and then pick the one that holds.

Social behavior appears even without being asked for

Finally, the authors check whether such behavior arises on its own under RL, when the model is rewarded only for the correct answer. On a small base Qwen-2.5-3B they run PPO training and see a curious dynamic: as accuracy climbs, questions-and-answers and conflicts of perspective turn up in the traces more and more often. Which is to say, the social organization of thought may be something RL finds as an effective strategy.

And if you prepare the model a little before RL — fine-tune it on examples where the solution is laid out as a short discussion between several roles — progress speeds up noticeably more than with fine-tuning on monologue chain-of-thought. That is an important practical conclusion: it is not only the data of correct answers that decides the outcome, but the form in which the model learns to think.

What this changes in how we see reasoning models

Reasoning models are not stronger simply because their reasoning is longer. They appear to move systematically into an internal multi-agent arrangement within a single stream of text — with no explicit multi-agent system of several models behind it. And argument, verification, changes of position and even a little emotional role-playing turn out to be a way to improve the search and cut the chance of a miss.

In practice this points to a new lever on quality: you can design the prompt, the training and the test-time compute so that the model enters internal deliberation more readily. And interpretability gets a more concrete target: not abstract logic neurons, but the mechanisms that switch on and coordinate different perspectives.