Worldmodeldata is betting game footage can train AI world models

The data problem world models need to solve

Large language models learn from text. World models need something different: examples linking what a system sees with what happens after it acts. That kind of cause-and-effect data is scarce online, says Xiaotian Zhu, an AI associate professor at the University of Surrey.

Researchers including Fei-Fei Li and Yann LeCun have focused on world models as a possible route beyond systems trained mainly on words. A model that might control a robot arm, for example, needs to connect a visual scene with details such as how hard to grip an object or how much force to use when moving it.

Some labs gather this data themselves, using people and robots to perform tasks in test environments while sensors record the results. The method produces limited volumes and may miss unusual situations. Nicole Frenkel, a partner at Khosla Ventures, which invested in General Intuition, says repeated demonstrations of picking up and moving an object do not capture the disorder of the real world.

Worldmodeldata’s plan is to aggregate gameplay data for research labs, rather than make each lab negotiate separately with game studios. The startup joins companies such as General Intuition and Niantic, which collect game data on their own platforms.

What a million hours can—and cannot—show

Worldmodeldata says it has licensed nearly one million hours of gameplay data from studios behind popular video games. CEO Rea Lucas did not identify the studios. The company plans to let individual players receive compensation if their data is used for training.

Lucas argues that games increasingly resemble the real world, and that varied gameplay could expose models to important edge cases. Frenkel points to the stakes in settings such as cars, aircraft, drones, factory forklifts and quadruped robots, where a mistake can be costly.

Lucas expects game data to make up a large share of world-model training, followed by fine-tuning on data from a specific real-world environment or task. She sees the possibility of a “GPT moment” for world models: a point at which they become genuinely useful.

The unresolved issue is whether breadth can make up for a mismatch in physics. Nvidia, which makes a family of world models optimized for its chips, uses its own engine, designed to reproduce real-world physics. Min-Yu Liu, who leads world-model development at Nvidia, doubts models trained on game data will perform well at precise movement control, such as carefully handling objects.

A character may reach for an apple, but the game may not model how each finger must grip it to keep it from slipping. Liu says game data is better suited to world models that generate highly realistic video or 3D environments. Zhu likewise describes games as simulations that rely on physics only approximately.

I think the million-hour figure matters less than what those hours actually record. The announcement does not say how much detail the licensed data contains about forces, contact or object handling—the very information that distinguishes a convincing scene from a useful training example. If games can supply variety but not reliable physical consequences, they may help models imagine worlds without teaching them how to act in one.