How an AI agent selects past experience for a new task

Memory That Waits for a Question

Imagine an AI agent that learns how to find an object in a house. Later, it’s asked to heat up some food. The earlier experience might contain useful details: the route to the object, the sequence of actions, or how to change its state. But if the agent has already compressed that experience into a short note, some of those details may be gone. And it won’t know which ones mattered until the new task arrives.

The authors propose changing the order: keep detailed records of past experience, and turn them into a brief instruction only when the current task is known. That’s how JitMem works. It finds relevant past attempts, then asks a separate model—the memory curator—to choose what will help with the task at hand.

The authors tested the approach on tasks in a virtual home, an online store, and a customer-support setting. In all three, JitMem outperformed methods that turn experience into ready-made notes and skills in advance.

Why It Matters When Memory Is Processed

Most memory systems for AI agents decide what to save as soon as a task is complete. An agent might record reasoning about a failure, a useful rule, or a ready-to-use sequence of actions. Later, it retrieves a relevant note by matching it to a new task.

There’s a limitation to this approach: the note is created before anyone knows what it will be needed for. The same experience can offer different lessons. One task may depend on the order of actions; another, on how to move an object or change its state. A one-size-fits-all note may be too general, while a detailed one may contain too much.

JitMem puts off that decision until a new task arrives. It stores complete records of past attempts: the task description, the agent’s observations, and its actions. It then retrieves several similar records, and the curator uses them to write a short instruction tailored to the current request.

JitMem first retrieves past attempts, then creates an instruction for the current task and passes it to the agent.

For example, the same record of interacting with objects in a house can be used in different ways. For the task “put the hot potato in the refrigerator,” the curator might focus on actions related to heating and cooling. For “put the newspaper on the couch,” it might highlight how to find the right spot and check the result. The experience stays the same; the instruction changes.

This also addresses another problem: training the memory system itself. When a note is created in advance, its usefulness may not become clear until a relevant task comes along later. The reward is delayed, making it difficult to connect the decision to save a particular detail with success on a future task.

In JitMem, the curator prepares an instruction for the current task, and the agent tries it right away. If the task succeeds, that provides an immediate training signal. The authors use reinforcement learning to update the curator based on the outcome. The system doesn’t have to guess which tasks to group together in order to evaluate a note much later.

What Is JitMem?

The system has four parts: a store of past attempts, a retriever, a memory curator, and an executor—a frozen language model that solves tasks. Only the curator is trained. Retrieval is straightforward: it matches the current task description against descriptions of past tasks and returns the three closest matches.

The curator reads the current request and the retrieved records, then writes a short instruction. The executor receives it along with the task and acts in the environment. Once the task is complete, the system evaluates the result. Only successful attempts are added to the store. The instruction created for the current task is not saved; the original record of actions is.

To train the curator, the system generates several instruction variants for a task. The frozen executor tries each one. Successful variants receive higher scores, and the curator learns to produce similar instructions. The executor itself doesn’t change.

This separation offers a practical advantage: a trained curator can be paired with other executor models without retraining. In the tests, a curator trained with Qwen3-8B came close to one trained with GPT-5.4, with a gap of just 1.4 percentage points on one test.

What the Tests Showed

The authors evaluated JitMem on three benchmarks. In ALFWorld, the agent performs household tasks in a text-based environment. In WebShop, it selects products from an online store. In τ²-bench, it talks with customers and uses tools to handle requests in the airline, retail, and telecommunications sectors.

In the main set of tests, trained JitMem with the Qwen3-8B executor completed 77.4% of tasks in ALFWorld and 32.8% in WebShop. The best method that saves ready-made skills from past tasks scored 61.2% and 16.5%, respectively—a gap of 16.2 and 16.3 percentage points. On τ²-bench, JitMem improved on the best comparison method by 3.9 points.

JitMem performs worse without key components: the curator needs the current task, complete records of past attempts, and a filter that keeps successful cases.

Notably, JitMem can outperform methods based on prewritten notes even before reinforcement learning. In one WebShop comparison using Gemini-2.5-Pro as the executor, JitMem achieved a 61% success rate, compared with 41% for SkillOS, when the same Gemini model served as both curator and executor in each setup. This suggests that some of the improvement comes from the core idea—tailoring memory to the task—rather than simply using a more capable model.

Training brings additional gains. When the curator no longer sees the current task, performance drops noticeably. On WebShop, the drop reaches 10.4 percentage points. Performance also falls if past records are removed and the system is left with only the model’s own knowledge. This suggests the curator learns to draw on experience, rather than simply offering general advice based on what it already knows.

As training progresses, the success rate rises while the number of executor steps gradually falls.

Component tests help explain where the gains come from. Including failed attempts alongside successful ones reduces performance. So does replacing full histories with short, ready-made notes—especially in WebShop, where performance drops by as much as 8.2 points. When a short note is prepared in advance, some details are lost for good.

JitMem can also reduce context length and the number of actions. In a separate test with GPT-5.4, it used 50–56% fewer input tokens than methods based on notes and skills, and required 28–31% fewer executor steps. A shorter instruction doesn’t mean less useful information: the curator keeps the details that matter for the current task.

Where the Approach Is Still Limited

The system adds a separate language-model call to prepare the instruction, increasing both cost and response time. Retrieval also relies on matching words in task descriptions. That method is simple, but it may work less well as the memory store grows and tasks become more varied.

There are also questions about how well the results carry over to real-world use. The tests were run in three predefined environments, not across a broad stream of everyday tasks. For τ²-bench, the authors tested an untrained version because no standard training set was available. It’s not yet clear how well a trained curator would transfer to more open-ended, changing scenarios.

Finally, the instruction format is set separately for each environment. The curator uses different system prompts for household tasks, shopping, and customer support. The authors also identify better retrieval and instruction-format selection as open areas for future work.

Conclusion

An AI agent can keep detailed records of its experience and select the relevant parts only after it knows what the next task is. That’s the central idea behind JitMem. It reduces information loss when experience is stored and lets the curator learn from immediate results, instead of waiting many tasks to find out whether an old note was useful.

For now, the results are limited to specific benchmarks and need further testing on new tasks. But the principle could apply more broadly: the same past attempt can be used in different ways, depending on what the agent needs to do now.