Case-based memory lets an agent improve without touching its weights

When we ask a large language model (LLM) to solve a hard problem, one well-crafted prompt no longer carries the job. In practice the work is a sequence of actions: search, read, write code, check, fix. The agent has to plan its steps, use tools and remember what it did before. Yet most agents today are either hardwired into rigid scripts that adapt badly to new conditions, or they demand expensive fine-tuning that retrains the weights. Can an agent keep improving continuously and cheaply, without touching the base model's weights?

AgentFly says yes, and bets on memory. The idea is simple: the agent stores its past attempts — the successes and the failures — as cases, and when it meets a new task it retrieves the similar ones and steers by them. This is case-based reasoning: we solve something new by recalling something old and familiar. Formally the authors frame all of it as a memory-augmented Markov decision process (M-MDP), in which the case-retrieval policy is learned online. Nobody fine-tunes the base LLM: what grows is the memory, plus a lightweight layer on top that picks the right cases.

How it works under the hood

The architecture is planner–executor. The planner (an LLM) looks at the task, pulls K relevant cases from a shared pool of previously solved problems, and writes a concrete plan: which subtasks to do and with which tools. The executor (a second LLM) acts: searches the web, walks pages, writes and runs code, reads documents and images through MCP, the protocol that ties the various external services together. When the run ends, the successful (and useful) steps are added to the case bank — the memory grows, and with it the odds of winning the next one.

Why it works

The subtle part is not simply recalling something similar, but picking the right past experience. That is handled with soft Q-learning: the agent learns to score how useful a case is in the current state. There are two variants. The non-parametric one takes the cases nearest in meaning (by cosine similarity of embeddings, for instance). The parametric one trains a lightweight function Q(s, c) that predicts whether case c will help in state s. In both modes the base LLM's weights stay frozen, and adaptation happens through reading and rewriting memory.

What the benchmarks showed

On benchmarks where multi-step plans and tools matter, AgentFly posts strong numbers. On GAIA validation it reaches 87.88% Pass@3 (first place); on the test split, 79.40%. On DeepResearcher, mean F1 is 66.6% and Partial Match 80.4%, ahead of even the trained systems. On SimpleQA accuracy is 95.0% — a new bar for web agents. On Humanity's Last Exam (HLE) AgentFly scores 24.4% PM and takes second place, nearly catching the closed next-generation models.

What changes as memory grows

What is interesting is that the payoff comes from a small but high-quality memory: raising K (the number of retrieved cases) improves quality up to roughly K=4, after which the effect saturates. On out-of-distribution (OOD) tasks, case memory adds 4.7–9.6 percentage points — so analogies really do help carry strategies over to new domains. And on the hardest tasks the bottleneck turns out to be not the length of the answer but the volume of input context: plans, tool protocols, intermediate reasoning.

Practical observations

• A fast planner wins more often. In the experiments, the quick GPT-4.1 as planner consistently beats the more deliberative models: a short, structured plan does better than a long thread of reasoning with no clear decomposition.
• Tools are not a cure-all. Where the model has little background information, tools and planning together give the biggest lift; but on the open web, too much external information without memory can throw the agent off. AgentFly handles that through memory and case selection.
• Curation matters. The authors filter what goes into the case bank, writing down the final steps above all rather than the entire noisy trace. That saves tokens and makes retrieval more useful.

Where this leads

AgentFly shows that adaptive agents can be built without heavy fine-tuning. Memory makes learning continuous: the agent carefully accumulates experience and puts it to work in new tasks immediately. That is a route to sturdier and more affordable systems — from enterprise search to research assistants, in settings where retraining the model is out of reach on cost, latency or security policy.

Questions remain for the next step: how to automate memory hygiene (compression, deduplication, privacy), how to share cases between agents in a multi-agent system, how to account for causality in case selection rather than semantic proximity alone. But the main thing is already clear: tie planning, tools and a living memory together and you get an agent that learns the way a person does — not by rewriting its brain, but by adding to its notebook intelligently.

💾 Code