Rewriting an agent's context collapses it; small edits gain 17 points

Over the past two years one thing has become clear: many applications built on large language models learn better through careful work with the context than through fine-tuning weights. Into the context go system instructions, reasoning steps, examples, domain rules, facts, even hints on how to use tools. It is transparent, portable, and it works at runtime. On top of that, progress on long context and KV-cache reuse keeps making this path cheaper and more practical.

But two unpleasant traps come with it. The first is brevity bias: optimizers, and the models themselves, tend to squeeze instructions down to generic phrasing and forget the fine domain heuristics. The second is context collapse: when everything is rewritten monolithically, the model crushes accumulated work into a short summary and quality falls. On AppWorld the authors watched a context of 18,282 tokens at 66.7 accuracy turn into 122 tokens at 57.1 — worse than the plain ReAct baseline at 63.7.

The idea: context as an evolving playbook

The authors propose ACE — Agentic Context Engineering. The idea is to treat the context as a living playbook that grows, gets refined, and is not afraid of being long. Unlike people, models are fairly good at filtering for relevance on the fly, so a detailed context works for them rather than against them. ACE's core principle is not to rewrite everything from scratch but to make careful edits and clear out duplicates on a regular basis. That is what avoids collapse and keeps the domain-level details.

What ACE actually does

Three roles inside ACE

• The Generator collects new trajectories and marks what worked and what did not.
• The Reflector pulls lessons out of successes and failures, turning experience into rules a reader can follow.
• The Curator writes those lessons into the context as compact delta updates and merges them without breaking the structure.

The context is stored not as a monolith but as a set of bullets with metadata: an identifier, usefulness counters, and the content itself (a strategy, a common error, a domain hint). Running on top of that is a grow-and-refine loop: grow when a valuable new piece shows up, and periodically drop duplicates by semantic similarity. The result is a playbook that stays large while remaining manageable and interpretable.

What it looks like in practice

On agentic tasks like AppWorld, ACE updates system prompts offline and agent memory online. On the financial benchmarks the playbook covers domain formats, extraction templates and the common pitfalls of XBRL. Importantly, ACE can run without labeled data: it leans on natural execution signals — code that passes or fails, API statuses, answer matches.

What the experiments showed

• AppWorld. Offline ReAct+ACE lifts the average to roughly 59.4% (+17 points over the baseline) and lands level with the leader, IBM‑CUGA, even though it runs a smaller open-source model. Online ACE shows its strength on the harder tests: average gains of up to +17.1, and on the challenge split it goes ahead of IBM‑CUGA.
• Finance. On FiNER and Formula, ACE delivers steady gains. Offline with labels it reaches an average of 81.9, without labels 77.1. Online ACE wins as well, particularly where the feedback signal is good.
• Cost and latency. Incremental deltas and merging without an LLM in the loop save resources: up to −82–92% on tokens and rollout latency.

Why it matters

ACE shows that a rich, growing context is not the enemy of efficiency. Handle the KV cache properly and long playbooks get cheaper to maintain, while the investment pays for itself through steady quality gains. That is worth the most in continuous settings, where fine-tuning weights is expensive and risky. The context stays interpretable, it is easier to edit, easier to unlearn selectively, and easier to hold to privacy and audit requirements.

Where the method runs out

ACE depends on how strong the Reflector is. If it cannot extract useful lessons, the context fills with noise. In very narrow domains, where the model cannot turn experience into knowledge, the playbook simply will not grow. And not every task needs a rich context: sometimes one or two strategies are enough. ACE pays off most where domain detail, complex tools and long chains of actions matter — a multi-agent system interacting with an environment, say, or XBRL reporting.

The short version

This work removes two systemic barriers to context adaptation: the pull toward brevity, and collapse. In their place it offers an engineering approach — modular deltas, reflection, growth paired with cleanup. The result is self-improving LLM applications that learn quickly, cost less to run, and hold on to the details.