Scoring the whole session makes a shopping agent behave more like a user

A recommender system can predict the next click perfectly and still barely understand the user.

A shopper scrolls the catalog, goes back to an item they already looked at, compares two similar models, puts the purchase off, then suddenly adds something to the cart. A session like this is full of pauses, repeated actions and stray movements. Train a simulator on the next action alone and it will copy individual clicks while losing the logic of the behavior as a whole.

The authors of RecVerse propose treating the shopper as a participant in a long process. An AI agent has to see the screen, remember how the session has gone, sharpen its picture of the user's interests as it goes, and be scored on the whole trajectory. To train it they also released USB, an interactive dataset of real shopping sessions.

Why simulating a shopper matters

A user simulator exists so that recommender systems can be tested without constant experiments on real people. It can be used for:

🟠 checking a new product ranking before launch;

🟠 evaluating hypothetical interface changes;

🟠 training a recommender system with reinforcement learning;

🟠 analyzing how recommendations affect clicks, add-to-cart events and purchases.

The problem is that a realistic shopper is not a function of the form “show an item → get a click.” Behavior depends on the entire session history. The user may have seen the item they wanted ten screens ago, compared it with something else, and come back to it after reading reviews.

Earlier systems handled this in one of two ways. They either discarded old observations or stuffed the whole history into the model's context. The first approach erases long-range connections. The second inflates the context window fast and does not always help: the authors' experiments show that past a certain amount of visual history, quality starts to fall.

Quality rises at first along with the volume of visual memory, then plateaus and dips slightly, while the cost of the context keeps climbing.

There is a second problem. Training on each individual action pushes the agent to reproduce behavior logs too literally. It may end up browsing products without ever stopping or, at the other extreme, doing almost nothing. Either way the average session looks nothing like a real one.

What RecVerse is

RecVerse is a vision-based AI agent that simulates a shopper in a mobile storefront. At every step it receives a screenshot and a list of available actions. The model then solves three tasks at once:

🟣 which action to take in the interface;

🟣 what current intent to attribute to the user;

🟣 what to write to memory.

The action set includes scrolling, tapping an item, opening a product page, going back, adding to the cart, buying and ending the session. Memory counts as part of the action too. The agent decides for itself when to log an event and when to update its general picture of the user's interests.

The architecture splits memory into three levels:

🟣 Working memory holds the last few screens, actions and current thoughts. This is short-term visual context: which products just caught attention and what the user is comparing right now.

🟣 Episodic memory records the events of the current session. For example: the user opened a sneaker page, checked the price, went back to the feed and moved on to a similar model.

🟣 Preference memory compresses the history into broader conclusions: the user is looking for inexpensive kitchenware, prefers a particular color, or avoids a specific attribute.

RecVerse ties the screenshot, the three memory levels, the interface action and the memory update into a single loop.

The arrangement echoes human perception. The recent screen is what the next decision needs. Session events keep the thread from being lost. Generalized preferences explain why the user picks some products and ignores others.

The key detail is that memory is not filled in by hard rules. The agent learns what to keep. It may write nothing after a routine scroll but log an event after a close look at a product or an add to cart.

Why training on the whole session works better

RecVerse is first fine-tuned on real trajectories by imitation. That gives the model a basic grasp of the interface and the legal actions. Then reinforcement learning begins.

Instead of scoring each click, the authors score the entire session at once, using three signals.

The macro level covers the overall rhythm of the behavior. How many clicks, product-page visits, add-to-cart events and purchases were there? How close are those numbers to what real users do?

The micro level checks the content of the choices. If the user was shopping for shoes, the agent gets credit for picking products in nearby categories, even when it did not open exactly the same item.

The third signal checks the format of the response and whether the chosen action can actually be executed in the current state of the interface.

Together these scores do different jobs:

🟠 the macro reward keeps the agent from becoming hyperactive or too passive;

🟠 the micro reward holds it inside the area of the user's real interest;

🟠 the format check stops the model from generating actions that cannot be executed.

For categories the authors use a three-level hierarchy. A match at the “clothing” level earns partial credit; a match at the “shoes → sandals” level earns more. That is more practical than demanding an exact product match: in a real store a person can pick a different item in the same category and still behave plausibly.

USB: an environment for real interaction

USB contains 5,274 real user trajectories, nearly 70,000 actions, more than 90,000 products and 5,222 users. Every session comes with page screenshots, a user profile, an interaction history and action labels.

The average trajectory is 13.24 steps long. The catalog is split into 41 top-level categories, 517 second-level categories and 2,256 fine-grained ones.

The distribution of USB products across category levels shows clothing dominating alongside a long tail of niche interests.

What sets USB apart from ordinary click logs is that it is an interactive environment. The agent can take an action that never occurred in the original session and still get the matching screenshot. That makes it possible to actually run multi-step trajectories and train the model with reinforcement learning.

In a static log, any deviation from the recorded action is a dead end: there is simply no next state for the new path. In USB the state of the interface is reconstructed from data about which products were shown and how the elements were laid out.

The dataset is also noticeably skewed toward clothing. That is natural for a commercial service, but it limits what can be concluded about other kinds of shopping. Someone choosing a dress may behave differently from someone shopping for groceries or electronics.

Results

The authors compared RecVerse with text-based and vision-based methods. Two groups of metrics were used.

The first measures how closely the overall style of behavior resembles the real thing: trajectory length and the rates of clicks, product-page visits, add-to-cart events and purchases.

The second measures how well interests line up. A system scores points for exact product matches and for category proximity.

Ordinary text agents sometimes posted good product-match numbers, but they got there by being far too active. They clicked and added items to the cart much more often than real users do. An agent like that can look accurate on individual metrics while its sessions look nothing like a human's.

RecVerse-GUI with reinforcement learning came closer to the real rhythm of behavior and matched user intent better at the same time:

🟣 the product-match score rose from 4.27 for the best vision-based baseline to 7.19;

🟣 the share of trajectories with at least one product match rose from 5.92 to 10.45;

🟣 hierarchical category matching went up from 23.11 to 32.64;

🟣 the click rate reached 7.78% against 9.08% for real users;

🟣 the average session ran 16.25 steps against 13.47 for real users.

The methods compared against real-user statistics: RecVerse sits closer to the region of realistic behavior and does not make up for a weak grasp of the user with excess activity.

That last number shows the limit of the approach. The model understands interests better, but its trajectories are still longer than a person's. It keeps exploring the catalog and does not always end the session when it should.

What the ablations showed

Removing individual memory components made the results worse. Working memory on its own is not enough: it helps recall the last few screens but does not preserve the overall line of interest. Dropping preference memory hurts the alignment between actions and user intent most of all.

The reward ablation told a similar story. Without the micro reward the agent is worse at picking products in the right categories. Without the macro reward the overall rhythm becomes less realistic.

Removing memory levels and individual rewards degrades quality: preference memory carries the long-term interest, and the micro reward carries the precision of the choice.

Humans rated the trajectories as well. In 74% of comparisons they preferred the real user to the RecVerse trajectory. For the vision-based baseline that figure reached 98%. Put the two simulators head to head, and RecVerse was chosen 92% of the time.

The gap with humans remains. Observers can still tell machine behavior from human behavior. Matching statistics and categories, then, is no guarantee of a natural session. People notice small things: odd pauses, repeated navigation, behavior that is too consistent, or a purchase that ends unconvincingly.

Scaling the model from 2 billion parameters to 4 billion improved interest alignment first and foremost. Precision, recall and category matching all went up, while the overall style of behavior stayed roughly as realistic as before. But model size did not fix every discrepancy: some browsing and purchase rates still differ from human ones.

The takeaway

RecVerse offers a way to model users. The unit to model is the whole session. For that the agent needs:

🟠 a bounded short-term memory for the current screen;

🟠 an episodic memory for the events of the session;

🟠 a preference memory for long-term intent;

🟠 reinforcement learning scored over the whole trajectory;

🟠 separate signals for the overall rhythm and for the content of the choices.

How much the approach is worth in practice depends on how well it transfers to real recommender systems. For now training is limited to short trajectories of up to 20 steps, and the data comes from a single marketplace and mostly describes clothing.

The direction, though, is clear. A simulator has to resemble a user in more than click counts. It has to remember what it has already seen, shift its interests as the session goes on, make mistakes sometimes and stop at the right moment. Only that kind of AI agent can become a useful environment for testing recommender systems before they are put in front of real people.