Agents that share latent thoughts instead of words reach 93% on MATH

Put several models in a multi-agent system on the same question and they will argue a little, correct each other, and settle on a compromise that is not always right. Language is what lets them do it, and it is also the bottleneck: it is sequential, often ambiguous, and rarely a faithful record of the reasoning behind it. A new paper argues for working above the level of words and giving agents a shared channel for those internal meanings themselves.

Why words get in the way

In an LLM-based multi-agent system, every agent carries its own goals, hypotheses and intermediate computation. Flattening all of that into text loses nuance, introduces mismatches and blurs the shared picture. The paper's core idea is to get agents to exchange compact internal states — thoughts — that carry both shared and private information.

How the paper formalizes it

The authors posit a single latent thought space Z_t and an unknown invertible map f that generates the agents' observable internal states, H_t = f(Z_t). Which thoughts influence which parts of a state is set by the sparsity of f's derivatives. That structure is what tells apart the components shared across any subset of agents from each agent's own private thoughts.

The key contribution is a theoretical identifiability guarantee for individual thoughts. Working nonparametrically, with no labels or other supervision, the authors prove that the thoughts shared between any agents can be recovered without being mixed into other latent factors, along with each agent's private thoughts and the global structure of who shares what with whom — all up to a permutation of coordinates. In plainer terms: the algorithm does not just return some vectors, it comes with a guarantee that the shared pieces really are shared by exactly the agents that need them, and are not contaminated by private ones.

What ThoughtComm looks like in practice

Out of the theory comes a practical way to extract thoughts from vector-valued internal states, packaged as the ThoughtComm framework. Round after round it pulls a common pool of thoughts out of the concatenated states of all agents using an autoencoder with sparse sensitivity regularization, works out which dimensions of those thoughts each agent needs and how they are shared across agents, returns to every agent only the components useful to it, sorted by how widely they are shared, and then injects them into the embeddings through prefix adaptation.

A few details the authors stress. Jacobian sparsity pushes the model to keep the links between thoughts and states minimal and legible — each thought acts only where it has to. Prefix injection is a compact, flexible way to hand a thought to the generator without expensive fine-tuning of the LLM. The overhead of the injection and the autoencoder scales with the dimensionality of the thought-communication embeddings, not with the LLM's total parameter count, so moving to larger underlying models does not cost more.

What the experiments show

The experiments start on synthetic data, to check whether the system really does separate shared from private. Recovery matches the true components closely across a range of dimensionality settings, and the main method beats an ablated variant with no sparsity.

Two further experiments use the MATH and GSM8K math benchmarks. Several agents discuss a problem for two rounds and then produce a final answer. The comparisons run from single-inference baselines at one end to the expensive extreme where multi-agent interaction is implemented by fine-tuning the LLM. The authors track not only final accuracy but also agreement — whether the agents converged on the same answer. On both benchmarks their system wins on both counts: with Qwen 3-1.7B on MATH it reaches 93% accuracy, 17 points above a strong baseline and more than double what a single model manages.

One technical point they highlight is robustness to prefix size: stretching the injected prefix from 1 to 16 shifts performance by less than 5%, and a single prefix vector already delivers nearly all of the communication benefit.

The system also holds up as the number of communication rounds grows. The researchers note that other systems sometimes hit a strange corner case where agreement climbs while overall performance drops — the agents are being talked into consensus by some other route. That pattern does not appear here: agreement actually tracks correct answers.

They also study the communication bandwidth available for injecting thoughts, and find that widening the latent space helps up to a point, after which results decline.

Where it fits and what comes next

The paper closes by sketching where thought communication should pay off: wherever agents need to share the generalizations and premises behind problem solving, planning, probability estimates, world models and error diagnosis. The framework, they stress, does not depend on an interpretable internal state — contextual embeddings of text responses or visual features would do. It does require a small internal state, or a reliable surrogate, and access to it; and while the method does separate intent from nuance, it is not meant to capture every latent factor, only the ones that matter for communication between agents.

The takeaway

Taken together, the paper makes an elegant move: multi-agent systems stop relying on language alone and get a language of thought layered on top of it, with shared and private components provably distinguished thanks to Jacobian sparsity and pulled apart by a simple recipe combining a sparse autoencoder with prefix injection. In practice that buys higher performance, more agreement when agents trade messages, and system overhead that is easy to predict. Multi-agent systems may finally have a more honest language — the language of thought.