Bidirectional memory: how agents evolve by remembering past steps

Today's "deep" AI agents do more than continue text: they run searches, call tools, gather facts from different sources and work through hard questions step by step.

Short on memory, even smart agents stall

Today's "deep" AI agents can search, call tools, gather facts from different sources and work through hard questions step by step. In practice, though, an agent like this behaves like a very smart intern who forgets every morning what they did the day before.

That is exactly what MIA — Memory Intelligence Agent is about. The authors propose not to bolt memory onto an LLM agent but to rebuild the logic of how an agent learns from experience and changes its behavior mid-run. Judging by the results, the idea works: the approach improves flagship closed LLMs and lets a relatively small 7B model outrun larger ones.

This matters beyond research benchmarks. If we really want agents that can work on a task for weeks, look things up, avoid repeating the same mistakes and adapt on the fly, none of it gets off the ground without coherent memory.

What is wrong with ordinary agent memory

In agent systems, memory usually comes down to a simple recipe: store past trajectories — queries, intermediate steps, retrieved documents, answers — and then pull them into the context of a new task by similarity. It sounds reasonable. The side effects show up fast.

First, a long context is not the same as smart memory. The more old traces you feed the model, the likelier it is that the useful signal drowns in noise. Second, as the archive grows, storage and retrieval get more expensive. Third, an agent often needs more than a fact from the past — it needs procedural memory: what worked, what did not, which search route paid off, where it hit a dead end.

The authors put it bluntly: many existing systems amount to a weak planner that pulls mediocre memories out of a bloated archive and uses them to steer an under-prepared executor. Memory is nominally there; serious gains are not.

What MIA is: memory as a separate cognitive system

Instead of one model with a huge context window, MIA splits the job across three components: Manager, Planner and Executor.

The idea goes like this:

The memory Manager is the non-parametric memory. It stores compressed, structured "workflows" from past attempts rather than raw, sprawling logs. On multimodal tasks, images are first turned into text descriptions so that everything lands in a single storage format.

The Planner is the parametric memory. It is a separate agent that takes the current question plus the retrieved experience and builds a search-and-reasoning plan: where to go, which subtasks to solve, in what order.

The Executor works from that plan: it calls tools, takes search steps, reads results and assembles the answer.

In MIA, memory does not go straight into the executor's head. It first shapes the strategy, and the strategy drives the actions. On complex multi-step tasks that is far closer to how human problem-solving works: sketch the route first, then move.

How MIA retrieves and stores experience

A key idea in MIA is that it keeps not only the "good" examples but the negative experience too. The system can pull up a failed trajectory as well as a successful one — as a warning about what not to do.

Every memory carries three signals:

The planner then takes that set and generates a step-by-step plan. The executor tries to carry it out in ReAct fashion: think, call a tool, look at the result, move on. If it runs into a wall, the Planner can fire the Reflect-Replan mechanism once — rethink the situation and issue a new route.

Once the run ends, the trajectory goes back to the Manager, where the long log is compressed into a compact workflow. That gives you a two-way link: memory helps build the plan, and new plans and trajectories update memory.

Why reinforcement learning was needed here at all

One of the paper's strengths is that the authors did not stop at the architecture — they separately trained its parts to cooperate, using two-stage alternating RL training built on GRPO.

The executor is trained first. The goal is to teach it to actually follow a plan, use tools correctly and recognize when the strategy needs revisiting. The executor is not supposed to be an "LLM with search"; it has to become an agent that can work to instructions.

Then the executor is frozen and the planner is trained. Now the planner learns to build better plans on top of memory and to judge more accurately when reflection and replanning are called for.

Rather than hoping the two models will somehow sort it out between themselves, the authors train them in sync but in turns. One learns to be good hands, the other a good head.

The most ambitious part: learning at test time

The most interesting part of the work is test-time learning. In most systems, inference and training are strictly separated: train first, then use. MIA proposes something more alive: while new tasks are being solved, the planner keeps updating on fresh experience.

Crucially, only the planner updates; the executor stays frozen. That lowers the risk of shaking the whole system loose and keeps execution stable. Two processes run at once:

So the agent does not simply shelve its experience — it digests it internally. That is the evolution memory systems usually lack: memory stops being an archive and becomes a mechanism of development.

Results: memory that actually helps

On multimodal benchmarks MIA posts the best average score among open-source systems. The telling part is that many standard memory approaches — RAG, Mem0, A-Mem — often come out worse than running with no memory at all. That is a strong and uncomfortable conclusion for the industry: bad memory can do more damage than no memory.

What sets MIA apart here is that memory works through planning rather than as an uncontrolled dump into the context window. Across seven multimodal datasets the system consistently beats previous memory baselines. On text tasks the picture is the same: a clear margin on HotpotQA, 2Wiki and especially GAIA.

One more effect that reads almost as a provocation: Qwen2.5-VL-7B inside MIA outperforms Qwen2.5-VL-32B without that memory architecture. That is an important signal for the market. We usually try to buy quality by making the model bigger. This work shows that organizing memory and planning well is sometimes worth more than a few extra billion parameters.

MIA improves even the top closed models

The authors checked separately whether the approach carries over to strong closed models — GPT-5.4, Gemini-3-Flash and Claude Sonnet 4.6. The answer: yes.

This is probably one of the paper's most practically significant results. Even a very strong model benefits from a separate layer of memory, planning and rethinking of its own actions.

What is especially interesting is that the gain for stronger models is usually smaller than for weaker ones, yet it holds steady. That makes sense: the better the underlying executor, the less easy headroom is left. But the durability of the effect suggests the architecture really is closing a systemic gap.

What the training and tool-use analysis shows

The authors did not stop at final metrics and looked at the training dynamics. RL gradually raises the reward and shifts the behavioral style of both planner and executor. Response length adapts to the dataset as well: on some the planner gets terser, on others longer. Indirectly that shows the model is not just memorizing templates but tuning its strategy to the type of task.

The tool-call analysis is useful too. It shows that systems without memory and with thin tool use have the worst accuracy. More important still: long-context memory combines badly with multi-turn tool use. When the agent has to run a whole chain of searches and checks instead of looking one thing up, simply stuffing the history into the context breaks down fast.

MIA wins here because it separates the roles: memory suggests the strategy instead of jamming the executor's channel.

What if there are no right answers? An unexpectedly strong section

Another strong idea is unsupervised self-evolution. In the real world an agent often has no ground-truth answer to tell it whether it succeeded. For that, the authors propose a multi-agent "peer review": several specialized LLM reviewers rate the logic, the reliability of the sources and the completeness of the result, and a separate agent issues the final verdict.

It is a smart move. Instead of a single LLM-as-a-judge, which readily mistakes nice prose for a correct answer, the authors break the score down into independent dimensions. The unsupervised version of MIA turns out to be not merely viable: in places it nearly matches the supervised variant and beats many fully supervised baselines. It also keeps improving over repeated passes through the data.

If that result replicates widely, it is a very important step toward agents that can get better without hand labeling after every episode.

What matters most in this work

The paper's main value is not that the authors came up with one more memory block. There is no shortage of those right now. The value lies elsewhere: they show fairly clearly what memory for deep research agents has to look like.

Not an archive of facts. Not plain similarity retrieval.

A system that:

That looks less like an "LLM with a memory module" and more like the seed of a more autonomous cognitive architecture.

Takeaways

MIA is one of those papers where a loud name covers a genuinely substantial engineering idea. The authors take the sore problem of agent memory and show that it cannot be solved by enlarging the context window alone. What is needed is a split of roles, compression of experience, learnable planning and the ability to update itself.

The results look convincing: the system improves open-source and closed models alike, does especially well on hard multi-step tasks and does not fall apart in unsupervised mode. The main conclusion, though, is broader than the paper itself: as agent systems spread, the next race may be run not only over model size but over how exactly a model remembers, plans and learns from its own experience.

And if that is right, MIA is not just a well-built framework but a good hint of where the whole class of deep research agents is heading.