i
Research
Review · 2026-11-20

How AI Agents Train in Games

Cover: How AI Agents Train in Games

Game worlds can be turned into testable learning tasks, and playthroughs into training data for visual AI agents. In Game2Player, players themselves test the games: their reports help fix the environments, and every action is tied to measurable progress. A model trained on this data is more likely to move toward its goal, even in unfamiliar games.

The main idea

Training AI to play takes more than video. It takes records with clear goals, actions, and reliable measures of success. Game2Player automatically prepares this data from web games and uses in-game progress to identify useful actions.

🟠 A collector adapts each game and creates tasks whose progress and success are measured using the game’s internal state.

🟠 Two players test the environment in different ways and report problems.

🟠 Once fixed, games yield trajectories: sequences of screens, actions, and progress scores.

🟠 Training gives more weight to actions that move the player toward the goal.

What is Game2Player?

Game2Player is a pipeline for finding, adapting, and testing games, collecting gameplay records, and training a model. The environment and the players improve each other: players find errors the collector missed, and the collector checks their reports against the playthrough record.

Game2Player prepares a game, tests it with two players, fixes reported problems, and collects training data.

🟠 Game finder selects games that can run offline in a browser, be played solo, and be restarted reliably.

🟠 Collector packages each game in a reproducible environment, sets the starting conditions, and creates tasks. For each task, it defines how to measure intermediate progress and what counts as success.

🟠 Image-based player sees a screenshot and chooses an action; the game then pauses until the next decision. It plays in a way that closely resembles how the trained model will act.

🟠 Programmatic player writes a separate program for each game. The program reads information about visible objects that is available to the player and acts in real time, without calling a model at every step.

🟠 Report review: Players report broken controls, incorrect progress scores, or insufficient observable data. The collector checks each report against the recording and the game state, then fixes confirmed problems.

🟠 Data selection gives more weight to actions followed by noticeable progress and less to repeated actions and final moves in failed attempts.

How it works: a task in a web game

Suppose the task is to deliver iron and buy a level-one drill. The system needs to record button presses and distinguish progress toward the goal from aimless play.

The programmatic player acts on observable data in real time, while the image-based player chooses an action from a screenshot; their records are converted to a common format.

First, the collector finds the game’s data for iron supplies and drill purchases. It uses these to define a progress scale: each delivery of iron raises the score, and buying a drill counts as success. The score is based on the game state, not on another model’s judgment.

The players then attempt the task. The image-based player sees the screen, reads the control instructions, and chooses an action, such as pressing a movement key. The programmatic player gets information about visible objects and can calculate more precisely when to press or release a key.

If iron is delivered but the progress score does not change, the player reports a possible error. The collector reviews the recording and checks whether it linked the progress metric to the right in-game event. If the review confirms a problem, the collector fixes the measurement and has the players try the task again. Things can still go wrong: a report may be mistaken, the game state may be misread, or the task itself may turn out to be impossible. So the collector must verify every report and check that the fix worked.

Once the environment is approved, playthroughs become training examples. The system pairs each screen and the preceding history with the action to take. Examples followed by progress are sampled more often during training; unhelpful repetitions get less weight. This teaches the model to choose actions that move the task forward.

How the results are evaluated

The authors assess both the quality of the game environments and the value of the data they produce. In the first test, they compare the full pipeline with a version that has no player reports and with a standard coding agent that lacks specialized tools. The tasks are set in advance, and the environments are checked against the original games; people also review the progress scores.

The trained model is evaluated on two benchmarks: Player Bench, with 50 games held out from training, and the external GameWorld benchmark, with 34 games not included in the training collection. Both measure progress—the average share of the path to the goal completed—and success—the share of tasks completed in full. The training games do not overlap with Player Bench, and GameWorld is also separate from the collection.

The comparison includes base models, models trained with Game2Player, and other vision models. Participants and models receive task and control descriptions, and each attempt is capped at 100 steps. One caveat: the game pauses while the models make each decision, so the test does not show how they would handle real-time delays.

In a study of 20 games, player reports increased the number of correctly implemented tasks and the number of tasks completed.

The numbers

568 games
turned into testable environments
3,436 tasks
each with a measured progress score and success condition
15.7K trajectories
playthroughs collected, with a combined duration of 307 hours
14.3% → 31.2%
Qwen3.5-9B’s progress on Player Bench after training
30.0% → 53.1%
the same model’s progress on the external GameWorld benchmark

The model improved substantially on both evaluations, including on games it had not seen during training. In a separate test of 20 games, player reports raised the share of correctly implemented tasks from 65% to 72% and increased the number of tasks completed by programmatic players from 34 to 46. Having players check the environments together helps collect more records and fix the conditions those records depend on.

Conclusion

Game2Player automates the collection of game data with verifiable goals. A programmatic player can run many times without a separate model call at every step, while the image-based player adds decision-making examples that resemble how the future model will perceive the game. Progress-based weighting helps the model learn from actions that actually move the task forward.

Testing does not guarantee that every environment is error-free: some tasks in the study were implemented incorrectly after collection. The dataset is limited to solo games that run offline in a browser; it does not cover online games, multiplayer modes, or large commercial titles. Models also help build the game worlds and review reports, and individual versions of those models change over time.

For now, the results suggest this approach is useful for training and testing visual agents in games. The next step is to find out whether the gains hold in more complex environments, where agents must make decisions without pausing the game and account for real time.

AI reviews in simple way

Every day we read fresh AI papers and retell the essentials in plain human language — no hype, no fluff. If you want to see where AI agents are heading before everyone else, subscribe.

New reviews — every day

Follow on X