Natural-language memory beats fine-tuning on long agent tasks

Large language models do well on short reasoning and coding benchmarks. But real work stretches over dozens or hundreds of steps, demands switching between applications, careful context management and the ability to catch your own mistakes. The core problem is that at test time most agents stay static: they accumulate no experience and get no better from one attempt to the next. The authors of MUSE offer a simple but strong answer — give the agent a live memory and teach it to turn its own trajectories into knowledge it can reuse.

MUSE in one paragraph

MUSE is an agent built around a plan–execute–reflect–memorize loop. It works through a task step by step, assesses its own work after every meaningful subtask, turns successful action sequences into structured instructions and files them in memory. When the larger task ends, it distills more general strategies and tool-specific tips. Memory is stored in natural language, so it carries across models and needs no parameter fine-tuning.

Memory as the engine of experience

MUSE has three kinds of memory, and they work together:

• Strategic: short lessons about the dead ends the agent typically hits and the ways around them. These principles are loaded in as background context and periodically compressed so they don't bloat.
• Procedural: a library of SOPs — step-by-step instructions for subtasks inside specific applications. The agent keeps a lightweight index and pulls the details in on demand, so the context window never gets overloaded.
• Tool: muscle memory for tools, in effect. After each task, refinements land here that let the agent act faster and more reliably on the next attempt.

A separate reflection agent checks whether the goal was actually reached, using observations and artifacts, and either adds a new SOP to memory or writes up a failure report and triggers replanning. Crucially, SOPs come out of real successful trajectories, which is what makes them practical.

Where it was tested

The authors test MUSE on TheAgentCompany (TAC) — 175 office productivity tasks spanning HR, finance, project management and software development. More than 40 steps per task on average, often across several applications at once. The base model is Gemini-2.5 Flash; the NPCs in the environment run on GPT-4o. Actions per subtask are capped at 20.

What the experiments showed

• Continual learning: on a set of 18 tasks, the metrics rose monotonically across three consecutive passes with no human in the loop. By the final pass MUSE was more than 10% ahead of the memory-free version on both share of checkpoints completed and overall score.
• Generalization: the memory accumulated on medium-difficulty tasks was frozen and carried over to the hardest set. Even the base MUSE without memory was well ahead of the popular frameworks, and memory added noticeably more on top: up to 40.68% on checkpoints and 33.41% on average partial score.
• Full TAC (175 tasks): MUSE with a lightweight model set a new mark — 51.78% average partial score, 59.92% on checkpoints and 41.14% PCR. Previous strong systems sat around 43% on average partial score, so the gap is close to a 20% relative gain.

Two cases in practice

On a task that called for collecting feedback about an employee, the agent opened a group chat instead of running three separate conversations and gathered the answers faster, with no loss of quality. Strategies pulled from memory, in other words, do not lock the agent into a rigid script — they give it a base for flexible decisions.

The other example is creating an issue in GitLab. Partway through it turned out the required account did not exist, so there was nobody to assign the task to. The agent created the account itself, added it to the team and saw the process through, adjusting its plan and leaning on SOPs it had already worked out for navigation and checks.

Why it works

MUSE does not try to build an elaborate world model up front. It carefully files away verified fragments of experience: how to log in reliably, how to confirm a file was actually written, how to verify the result after a click. These small bricks transfer easily between tasks and even between different large language models. The authors show that natural-language memory carries over to another engine with no degradation, and sometimes with a clear gain. One detail matters a lot: reflection. Ablations show that quality drops markedly without it — both execution and the training signal for memory get worse.

Takeaway

MUSE is no magic wand for tasks that need deep multi-step search or heavy high-level planning. But where the work breaks into clear subtasks and the same mistakes keep recurring, experience helps prune the dead-end branches and shortens the path to a solution. The approach is cheaper than constant fine-tuning and steadier than pure RL in sparse-reward environments. One more advantage: the memory is easy to read, correct and extend.

MUSE shows how an agent can grow stronger in the middle of the work itself. Hierarchical memory, strict verification of results and careful distillation of experience deliver steady gains and a new level of quality on long tasks. And all of it with a lightweight model under the hood.

💾 Code