Teaching an agent to fold its own memory cuts context 92% by step 100

Tasks that require tool use and repeated web search tend to produce long trajectories that break most LLM agents: either the agent piles up the entire history, which is punishing to carry in context, or it compresses the entire history at every step, which means forgetting details that mattered.

The idea: proactive context folding

AgentFold trains agents to use their own long-term memory store with multi-scale state summaries. Building that mechanism is not a trivial piece of engineering. The authors propose a folding mechanism and teach agents to fold memory, deciding proactively when to fold, at what scale, and how. At every iteration the agent makes at least one choice: fold a small block of memory, or — once the current subtask is finished or the trajectory has hit a dead end — fold several large blocks at once.

How the loop works

Every agent response is structured: thinking, folding, an explanation, and a tool call. After the tool answers, the new observation is appended to the latest step, and the folds update the state summaries right away. The loop reads: perceive → understand → fold → act. Making memory an explicit step forces the model to work out what exactly from recent history will be useful next, and it cuts the noise that classic ReAct agents are so prone to accumulating.

Why this helps on long tasks

After the tool answers, the new observation is appended to the latest step and the state summaries update immediately. Each fold condenses a given range of state blocks into a single block. That working set has to contain everything the task needs. An explicit memory-folding step with multi-scale state summaries makes the model think about which parts of recent history it may need later, so the active working set ends up with less noise than in classic ReAct, and closed subtasks are not lost.

As a result, the average context length in use grows sublinearly over the course of a trajectory — for example, from 3.5K to 7K on BrowseComp by step 100, against a 128K token limit. That is 84K tokens shorter at step 100 (a 92% cut), saving roughly 7 GB of memory per run on one of the trajectories.

In one of the case studies, for instance, the agent notices at step 17 that steps 6 through 16 were a dead end, and performs a deep fold that collapses 11 steps into a single summarizing judgement.

Note that the training was SFT only: the authors assembled a dataset of trajectories with good actions and proper memory folding, discarding steps with environment errors and malformed steps, and then trained Qwen3-30B-A3B to answer the query with its reasoning.

How it does on the benchmarks

The upshot is that AgentFold-30B-A3B, using only its internal memory store, reaches 36.2% on BrowseComp, 47.3% on BrowseComp-ZH, 62.1% on WideSearch and 67.0% on GAIA (text only). That is above OpenAI's o4-mini (28.3 and 44.3 on BrowseComp and BrowseComp-ZH) and comparable to or better than GLM-4.5-355B-A32B and DeepSeek-V3.1-671B-A37B (the latter at 30 and 49.2). Raising the step limit on BrowseComp from 128 to 256 does not drop the final score — it raises it.

What this means for us

AgentFold trains agents to use a memory store with multi-scale state summaries instead of a copy of the whole history or a compression of the whole history at every step. Agents therefore get the benefit of both very long memory and strategies for using it.

The logical next step is reinforcement learning that rewards not just good folding but folding that maximizes the task's final reward. Given the scalability AgentFold already delivers, that brings agents able to operate over horizons of hundreds of steps in real business settings closer.