Letting an agent decide when to compress its own context beats a fixed threshold

When an agent runs out of memory mid-task

AI agents have a problem on long tasks: they get tangled in their own steps. Web searches, documents read, attempts, rollbacks, fresh hypotheses — all of it piles up in the context. At some point the history gets too long. The model either hits the limit or simply starts thinking worse because there is too much noise around it.

A new paper from Carnegie Mellon University and Meta offers more than another way to compress history — it reframes the problem. The authors call it ACM, agentic context management. The idea is simple: the decision to trim the history belongs to the agent itself, not to an external crutch bolted on around it. And it happens without losing the original data.

There is a lot of talk about million-token context windows, but in practice a long context does not cancel out overload. If you want an agent to spend hours hunting for an answer, writing code or working through a repository, it needs more than a large memory — it needs the skill to manage it.

What's wrong with ordinary context compression

The standard recipe goes like this: once the history grows past a threshold, an external module writes a summary, the old messages are thrown away, and the agent carries on from the short retelling. Better than nothing, but it brings two problems at once.

The authors set against this a model closer to human memory: a working memory for what is needed right now, and an external memory where the rest can be put away — but not forever.

The ACM scheme: a plain agent hits the limit, a summarizing agent compresses the history by force, and ACM decides for itself when to put context away without losing the original messages.

In ACM the agent has just two special tools:

The key point: the original history does not disappear. It simply moves out of the working window into external memory. So the compression here is effectively lossless.

What the authors actually propose

The paper's main idea is not compression as such but that context management becomes an action of the agent — a deliberate step like searching the web, opening a document or submitting a patch in a coding task.

The agent decides for itself:

This sounds natural, but for today's models it is not an obvious skill. Even very large models, the authors show, rarely start managing context well on their own. They have to be shown how and fine-tuned on trajectories of that kind.

How they are taught to do it

This is the most interesting part of the work. The authors did not just bolt two tools onto the model; they worked out how to teach it to use them at the right moment.

The training setup is a student-teacher pair. The student is Qwen3.5-9B; the teacher is the much larger Qwen3.5-397B-A17B.

The process runs in two passes:

But there is a subtlety that matters. The teacher makes two kinds of edits.

In the first, it looks at the trajectory without ACM and inserts the moment where context management should have been called — when the agent is looping, repeating the same searches, or has piled up too much useless history.

In the second, it looks at the trajectory with ACM and does the opposite: strips out compression that was not needed. So the model learns not only when to clear the context but also when to leave it alone.

The training-data pipeline: the student runs the task with and without the tools, and the teacher adds or removes context-management calls.

That is a sensible move. Telling an agent to compress the history more often is not enough — it will start doing it too early and get in its own way. Here the model is taught the balance: where a cleanup helps and where it breaks the line of the solution.

What it did in practice

The authors tested ACM on three long-horizon benchmarks:

The baseline comparison is against ReAct, a plain agent with no context management, and against systems with external threshold-triggered compression.

The results look like this: simply adding ACM already improves how the agent behaves, and fine-tuning on the purpose-built trajectories adds another clear gain.

For Qwen3.5-9B the authors report:

On BrowseComp-Plus, Pass@1 accuracy rises from 0.570 for ReAct to 0.727 for the fine-tuned ACM version. And the model stays an order of magnitude smaller than some of the best open and closed systems available right now.

Qwen3.5-9B accuracy on BrowseComp-Plus: ReAct, plain ACM, and three epochs of fine-tuning with ACM. Fine-tuning raises both mean accuracy and the consistency of the answers.

Another result that matters: ACM lowers peak pressure on the context. On average the peak token count drops by roughly 20% compared with a plain agent. That counts not only for quality but for inference cost: a lower peak means less load on server memory and on the model's cache.

Why the quality goes up

The authors analyze agent behavior separately, and the mechanism there is easy to follow. ACM does not merely save tokens. It lets the agent explore the task longer without cluttering the working window.

On the context-growth chart it shows up as a sawtooth: the agent periodically trims the history itself, before it hits the ceiling. Unlike ReAct, which only accumulates messages, ACM pauses to tidy up and then goes on searching.

Input tokens over the course of a solution: with ACM the context shrinks periodically, so the agent can work longer and does not run into the limit as fast.

Three effects follow from that.

The last point is the most interesting. The authors look beyond Pass@1 at metrics for agreement across several independent runs. ACM improves not only the ceiling of what the model can do but its repeatability. Put simply, the model does not just occasionally find the right path — it holds onto it to the end more often.

One telling example

The paper includes a vivid case: a multi-step question from BrowseComp-Plus that requires tying together several constraints drawn from an author's biography, a dissertation, a co-authorship and a restaurant's founding date. Without context management the base model failed in all four runs.

The ACM version behaved differently:

In that trajectory the raw history grew to 222,000 tokens, while the working window stayed below the base model's limit. The agent effectively solved a task that would have fallen apart long before the finish without external memory.

An example of a long ACM trajectory: the agent watches its own context size, alternates compression with memory queries, and covers a path that would otherwise have exceeded the window limit.

This is probably the most convincing passage in the paper. It shows the point is not a cosmetic tweak to the prompt but a new working habit for the agent.

What the comparison with big models shows

There is one more curious observation. Even GPT-5.5, inside this architecture, barely uses the context-management tools. A large model does not necessarily start clearing its memory sensibly just because the option is there.

That matters for two reasons.

First, context management is not a skill that arrives automatically as a model grows.

Second, distilling solutions from a strong model is not enough on its own. If the teacher solved the task without ever calling memory, the student never sees the behavior it needs. That is why the authors bet on purpose-built data in which the moments to compress and the moments to refrain are labeled explicitly.

A separate experiment confirms it, in fact. Plain distillation from GPT-5.5 does not produce the same gain as training on the ACM data itself. And a combination of the two approaches works best on part of the task set.

Where the approach runs out

The authors are straightforward about the limit as well: ACM is not useful for every model. If a model is too weak and gives up after two or three steps, there is nothing for it to manage. The context never has time to get long.

They give the example of a smaller model with 4B parameters. It almost always runs a single search, then guesses an answer quickly and ends the trajectory. It never reaches long-horizon problems at all. So the memory tools are useless there not because they are bad but because the underlying ability to sustain long reasoning is not there yet.

That is an important qualifier for the agent market. Context management is not a substitute for a model's capabilities but a layer on top of them. The agent first has to be able to work long and coherently; only then does memory start to help in earnest.

The takeaway

Long tasks break agents not only through the size of the window but through the mess inside it. ACM offers a simple, practical answer: make context management an action the agent takes itself, rather than an external timer with a summarizer attached.

Several things come through in the results:

If agents keep moving toward long autonomous tasks — deep search, working with code, complicated tool chains — mechanics like these will almost inevitably become a standard part of the stack. Not because context windows are small, but because even a large window has to be kept in order.