AI covers six roles in game development, but skills rarely transfer

AI in games is no longer just bots

When people talk about AI in games, they usually mean bots that learned to win at chess, Go or StarCraft. A new paper argues for a wider frame. AI in games today is not only the thing that presses buttons better than a human. It is also the thing that models the game world, shapes the story around the player, helps assemble prototypes, writes code, hunts bugs, and even changes the game on the fly while you are playing it.

The paper's central idea: AI in games should be discussed as a set of distinct roles across the entire lifecycle of a game. Do that, and two things become visible quickly. First, the roles really do exchange data and skills. Second, how far those skills carry is tightly limited by the specific game, engine, interface and audience.

This matters because the market and researchers often reach for the same story: if a model can handle one part of the pipeline, it will soon handle the rest. The paper cools that enthusiasm. Models do share a common foundation, but game tasks still run into the particulars of the environment.

Six roles for AI in the games industry

The authors sort the literature into six roles. The criterion is simple: what exactly comes out of the system, and what is it immediately used for.

Six roles for AI around a playable game: playing, modeling, design, development, in-play adaptation and testing.

🟠 Plays and acts — picks actions, builds plans, talks like a teammate or an NPC.

🟠 Models players and games — predicts player behavior and world dynamics, builds a world model or a simulator.

🟠 Designs games — proposes levels, rules, story branches, game objects.

🟠 Builds and maintains games — writes code, edits scenes, repairs the project, updates the build.

🟠 Generates and adapts on the fly — produces dialogue, quests, items and reactions during the session itself.

🟠 Tests and evaluates — plays through the game as a tester, looks for bugs, checks mechanics and quality.

The split looks obvious, but it earns its place. It keeps unlike achievements from being piled together. Inventing a mechanic, implementing it in code, and checking how it works for real players are three different jobs. In discussions of AI for games they tend to blur into one.

Where the progress is real

The short version: the field does best at two things, bounded play under clear rules and partially verifiable world models. It does worst at anything that demands long memory, durable adaptation and reliable operation in live production.

In the game-agent role, the picture is familiar. Some systems play a single game superbly. More general agents such as SIMA, NitroGen or Game-TARS try to carry behavior across games. But the paper keeps reminding you how easy it is here to see what you want to see.

An agent moving into a new game has not thereby understood the new rules. It may simply have carried over:

🟣 visual patterns — it recognizes doors, resources, enemies, the interface;

🟣 motor skills — it can move, aim, interact;

🟣 instruction following — it understands a task stated in natural language;

🟣 local habits — exploring the map, say, or collecting items.

But transferring rules, goals and the meaning of actions is a far rarer and harder thing.

The authors make a point of showing how much the interface decides. An agent handed a semantic API — "gather resource," "open inventory" — is facing one task. An agent with nothing but screenshots, a keyboard and a mouse is facing another. Comparing such systems on a single number is close to meaningless.

In short

🟠 Transfer between games often means transferred patterns, not understood rules.

🟠 The interface to the game changes what the task is.

🟠 A single metric explains almost nothing here.

World models: pretty is not correct

One of the paper's most interesting sections covers how AI learns to model the game itself. Not only classical planning, but the new generative simulators, where the world is literally drawn in by the model.

Research directions for modeling games and players: planning, simulation, state representation and long interactions.

Recent years brought systems like GameNGen, Genie, MineWorld and others that build interactive game worlds out of video and actions. You press a button, the model draws the next frame. It looks impressive. But the paper asks a very down-to-earth question: what exactly does such a system store, and what is it obliged to keep correct?

The distinction is a useful one:

🟠 Visual quality — the world looks plausible, motion is smooth.

🟠 Mechanical correctness — health drops the way it should, collisions work, items disappear once picked up.

🟠 State persistence — come back to a room a minute later and the key you already took has not reappeared.

This is where the trouble starts. A model can draw a beautiful picture and forget that the player is out of potions. It can stage a convincing fight and lose track of the boss's internal state. It can generate a smooth world that falls apart over a long run.

Examples of interactive simulators in which the model itself generates the game world in response to the player's actions.

If you want to train an agent inside such a world model, the error starts compounding. The agent finds a hole in the imprecise simulation, learns on it, then fails the task in the real game. The authors come back to this repeatedly: how useful a world model is depends on who consumes it. Action search needs one level of accuracy. Policy training needs another. A live player needs a third.

In short

🟣 A beautiful world is not yet correct mechanics.

🟣 Long-horizon memory of state is a separate problem.

🟣 A world model has to be judged by who uses it.

The player as something to model

The second branch of modeling is not the world but the human. Here too the paper draws an important line: predicting a player's next action, guessing their style, measuring engagement and improving their experience are not the same job.

A good example is the Maia line of chess models. They are not trying to play better than everyone. They are trying to predict the moves of humans at a given level, and even of a specific player. That is a different kind of task. Not maximum strength, but similarity to human behavior.

Which leads to a clear and routinely ignored conclusion: having a player model does not mean you have working personalization. Two things have to be checked separately:

🟣 how accurately the model describes the player;

🟣 whether the adaptation makes the game better for that player.

You can read the style well and still change the content badly. And the reverse: you can accidentally produce a more enjoyable level with no real understanding of the player at all.

The authors are especially careful wherever synthetic profiles, bots or the model's own judgments are used. A system that recovers artificially assigned traits is not thereby shown to understand live people.

From design to code: AI is already across the pipeline

The paper also traces how AI is entering design and development. This is no longer the fantasy of "generate me a whole game" but a set of quite practical roles.

In design, AI proposes levels, rules, story branches, 3D scenes. And here again the authors refuse to get carried away. Generating a plausible level is not enough. You have to check that it is completable, controllable, and matches what the designer asked for.

A good example is MarioGPT. It can generate levels from a text description, but those levels are then separately checked for completability and fidelity to the prompt. That is the recurring motif of the entire paper: generation almost always needs verification from outside.

Development is much the same. A coding agent can write code, wire objects into a scene, call the right methods. But the real problem in games is not the code as such — it is keeping code, scene, assets, collisions, engine interfaces and runtime behavior in agreement.

Directions for AI that builds and maintains games: code, scenes, coding agents, debugging and maintenance.

What is interesting today, then, is not the abstract "AI writes a game" but systems like Play2Code, AutoUE, UniGen, GameCraft-Bench. They take in a whole project, run it, try playing it, collect the errors, make fixes. The AI sits inside a closed loop: build it, run it, see the problem, fix it.

And here too there is no easy "the model has become a general-purpose developer." The authors stress that game projects are unusually full of bugs that compile cleanly and break on real interaction. Which means executable verification matters more than good-looking code.

In short

🟠 Generation almost always needs verification from outside.

🟠 A coding agent has to work with the whole project, not just the code.

🟠 Checks in an executable environment matter more than tidy-looking code.

On-the-fly generation and automated testing

The most fragile part of the whole field is the systems that change the game during the session. NPC dialogue, dynamic quests, difficulty adaptation, content tuned to the player's mood — all of it sounds natural for an LLM, and in practice runs straight into memory, latency and state consistency.

If an NPC promises an item, the engine has to actually hand it over. If the game reshapes a quest around your choice, the new branch must not break the rest of the story. If a companion answers out loud, the line has to arrive on time, not contradict the current state of the world, and not forget what happened five minutes ago.

Directions for AI that generates and adapts a game on the fly: dialogue, memory, dynamic rules and personalization.

Testing is uneven too. An automated tester has to do two different things: reach an interesting state and recognize that something went wrong there. These are not the same.

The paper shows why a strong player is not the same as a good tester. It can march confidently through the game and never enter the rare cases at all. Another agent will deliberately hunt for odd branches, break the usual route, and turn up more bugs.

One practical conclusion follows: coverage, behavioral plausibility and verdict accuracy have to be measured separately. Otherwise it is easy to end up with a tester that runs around a great deal and notices little. Or the reverse — an excellent judge that never reaches the situation that matters.

In short

🟣 On-the-fly generation runs into memory, latency and world state.

🟣 A strong player is not necessarily a good tester.

🟣 Coverage and judgment accuracy have to be measured separately.

What the paper says about transfer

The most useful part of the whole work is its honest treatment of transfer. The authors keep pulling two things apart:

🟠 reusing an artifact — game traces, a level layout, a player model, a test trace;

🟠 transferring a capability — when a system genuinely keeps a skill in a new game, on a new engine, or with a new audience.

That is the key distinction. One component handing another some artifact does not mean a shared competence has appeared. Game traces can be used to train a world model. A world model can be used to train an agent. A test trace can be fed to a coding agent. But at every next link you have to check again whether it works in the target environment.

Controllers, rules, world state, engine interfaces and player behavior all stay tightly bound to the specific setting. That is probably the paper's main conclusion.

In short

🟠 Passing an artifact is not transferring a capability.

🟠 Every transfer has to be checked in the target environment.

🟠 Game systems remain heavily tied to a specific game and engine.

The takeaway

If you want to understand where AI in games is actually heading, drop the simple picture of "there is a smart model, and soon it will make the whole game." The reality is more complicated and more interesting.

AI is already present across nearly the entire game pipeline: it plays, models the world, proposes levels, writes code, helps test, and adapts the player's experience. But each of these roles lives by its own rules, with its own metrics and its own failure modes.

The best-standardized tasks today are the ones with clear rules, a bounded interface and a legible check on the result. The worst are the ones that need long memory, accumulated change, repeated edits, stable world state and a real effect on live players.

The most practical thought is this: transfer between roles is possible, but it cannot be assumed by default. Any elegant pipeline has to be tested in the game, on the engine and with the players it will actually live in. Otherwise, instead of general-purpose AI you get a set of impressive demos that do not fit together.