Agents that swap hidden states instead of text get 8.3% more accurate

LLM-based multi-agent systems have an old and rather mundane problem: they talk too much. One agent writes a plan, a second critiques it, a third solves the task, a fourth calls a tool — and the whole collaboration bogs down in endless text generation, latency and token spend. On paper it looks like collective intelligence. In practice it looks like a long thread in a work chat.

When agents stop chattering and start thinking together

The authors of Recursive Multi-Agent Systems propose a sharply different move: what if agents don't have to exchange text at all? What if a system of several models could be turned into a single recursive computational loop, where the exchange happens in internal representations rather than words? That is RecursiveMAS — an architecture that lifts the idea of recursion from one language model up to an entire multi-agent system.

It sounds like pure engineering optimization, but it is really a bid for a new way to scale agent systems. Not just making agents bigger, but making them iteratively improve how they work together.

The idea: hidden states between agents, not text

The core claim is simple. If a single model can improve its answer by running computation repeatedly over its own hidden states, why not do the same for a group of agents? Each agent then behaves like a separate functional block in one shared recursive chain.

Instead of the familiar pattern — agent generates text, next agent reads the text, replies with text — the authors pass hidden states between agents. Not intermediate text, but the internal representation of meaning that the model has already computed and not yet decoded into words.

This matters for two reasons.

First, text is an expensive interface. Every intermediate step has to be decoded into the vocabulary, and the next model then re-encodes that text back into its own representations. It is slow and hungry for tokens.

Second, text is a poor medium for training the system as a whole. When a textual gap sits between agents, the useful training signal travels down the chain badly. The authors show theoretically that with text-based interaction gradients tend to vanish, while passing them through dedicated projections in the hidden space keeps them more stable.

In other words, hidden-state interaction is not only faster, it is also better suited to training.

How RecursiveMAS is built

The heart of the system is a module called RecursiveLink. It is a small two-layer residual projection that connects internal representations inside an agent and between agents.

It has two roles.

The first is the internal link. It lets an agent do more than read the original query: it can unroll its own chain of internal thoughts in the hidden space. The model takes a step, takes the last hidden state, passes it through the lightweight module and feeds it back to itself as the input for the next step.

The second is the external link. It carries representations from one agent to another, even when those are different models with different hidden sizes. Here the module does more than preserve semantics — it translates them between different representation spaces.

After that the system runs as a loop. The first agent generates internal representations and passes them to the second, the second to the third, and so on. The last agent returns the result to the first, closing the recursive loop. Only on the very last round does the system decode the outcome into text.

It is a small but consequential shift: text stops being the working medium of interaction and becomes only the display case for the final result.

How it is trained: warm-up first, joint optimization second

Take off-the-shelf LLMs and simply force them to exchange hidden states, and nothing good comes of it. So the authors split training into two stages.

First comes the inner loop: for each agent, the internal link is trained so that the agent's hidden thoughts line up with the way the model normally encodes correct answers. Put plainly, the agent learns to continue its reasoning directly in the hidden space rather than through text.

Then the outer loop starts: now the links between agents are optimized, and the whole system is unrolled over several rounds of recursion. After the final answer, the error is backpropagated through the entire loop, and the links between agents adjust to the shared result.

One point of principle: the base parameters of the LLMs themselves are never touched. All the large models are frozen, and only the lightweight connective modules are trained. That makes the approach far cheaper than full fine-tuning of every agent, and lowers the risk of wrecking their original abilities.

The result is an interesting trade-off. The authors don't try to retrain each member of the team. They train the team dynamics themselves.

Why this matters for more than speed

Agent systems today are usually discussed in terms of roles, communication protocols, tool calls and topologies of several agents. This paper asks a more fundamental question: can you scale the cooperation itself rather than an individual model?

This is where the work looks stronger than ordinary inference optimization. It proposes a new axis of scaling: not more parameters, a longer context window or more compute per agent, but more recursive joint processing inside the system as a whole.

That changes the perspective. Where a multi-agent system used to resemble a group of models sending each other messages, here it becomes something close to a single computational machine split into specialized blocks.

For the industry this is interesting for three reasons:

First, expensive text exchange is becoming the bottleneck more and more often, especially in complex pipelines with a critic, a planner, an executor and tools.

Second, if you can keep the strengths of different models and train only the seams between them, that opens a path to cheaper and more flexible systems.

Third, recursion here works as extra depth of reasoning, but distributed across agents. That is potentially more important than simply making one model bigger.

What the experiments showed

The authors test RecursiveMAS on nine benchmarks: math, science, medicine, search and code generation. And they check not one fixed architecture but four common styles of cooperation:

The headline result: against strong single-agent and multi-agent baselines, the method delivers an average of 8.3% more accuracy. At the same time the system runs 1.2–2.4× faster and cuts token spend by 34.6–75.6%.

What is especially interesting is that the gain grows with the depth of recursion. At three rounds of recursive processing the advantages become more pronounced in both quality and efficiency. You would normally expect extra iterations to cost more. Here they do cost compute, but the text baseline gets far more expensive, because every step pays again and again for generating intermediate answers.

Against baselines such as individually fine-tuned agents, TextGrad, LoopLM and a text-based recursive multi-agent system, RecursiveMAS comes out ahead almost everywhere. The gap is widest on tasks that demand deep reasoning: olympiad math, hard science questions, code.

There is another important finding: the approach is not tied to one particular agent topology. It works with a linear chain, with a mixture of experts, and with a fast student paired to a strong expert.

Where the efficiency gain comes from

The authors analyze speed and tokens separately. The logic is close to intuitive: if you don't have to push intermediate states through a huge vocabulary and back on every round, the system gets cheaper.

At three rounds of recursion, the token reduction reaches 75.6% relative to the text version. That is an enormous difference for practical systems, where the cost of communication between agents quickly exceeds the cost of the final answer itself.

There is a curious qualitative effect too. The authors visualize the distributions of final answers as the number of rounds grows, and show them gradually converging toward the distribution of correct answers. Put plainly, the system does not merely think for longer — it steadily corrects the direction of its representations.

Training cost is a separate advantage. Because the large models are frozen, RecursiveMAS needs fewer trainable parameters and less GPU memory, and works out cheaper than full fine-tuning while still producing a better average result.

Where the limits may lie

The work looks strong, but it does not come without caveats.

First, passing hidden states between different models is a finicky business. Lightweight projections do help align the spaces, but compatibility across even more heterogeneous architectures remains an open question.

Second, hidden-state interaction is harder to interpret. In a text-based multi-agent pipeline you can read the intermediate reasoning and see where an agent went wrong. Here a large share of the cooperation is buried inside vectors. That is good for efficiency and worse for debugging and auditing.

Third, the paper tests up to three rounds of recursion. That is enough to see the trend, but it is not the ceiling. How the system behaves under deeper recursion, and on longer tasks with heavy tool use, is still unanswered.

Finally, the approach leans heavily on how well the seams between agents can be trained. If those projections are poor, the whole idea can degrade quickly.

The bottom line

RecursiveMAS is one of those papers that looks at first like an architectural trick and on a second reading like an attempt to redefine the mechanics of multi-agent systems.

The authors argue for dropping the picture of agents as interlocutors trading lines, and seeing them instead as parts of a single recursive computation. That makes it possible to raise accuracy, speed up inference and sharply cut token spend at once. And above all, it gives a new vocabulary for talking about how agent systems scale.

In short, the paper's thesis comes to this: the next step for multi-agent LLM systems is not necessarily more agents, and not necessarily more text between them. It may be less text and more shared internal work.

And that is no longer just an engineering trick — it is a serious proposal for what the next generation of agent systems could look like.