OpenMobile-2 finds app variety matters more than training volume

Rebuilding the apps

Training on real apps can require accounts, costly purchases and a way to reset each app’s state. The team recreated 35 apps, including Taobao, JD, DiDi and Meituan, as well as Slack and Google Drive. Combined with existing simulations and apps in the Android emulator, the environment covers more than 110 apps.

Reviewers refined the recreated apps until their interfaces and workflows matched the originals. Orders, budgets and travel plans persist between sessions, information can move between apps through the clipboard, and each simulation resets automatically. The agent sees screenshots, not internal app data, approximating how a person uses a phone.

In hybrid mode, the agent combines normal screen actions with functions the app exposes directly as tools. | Image: Yan et al.

Source: the-decoder.com

The team used the environment to produce about 11,600 workflow examples and more than 2,000 practical tasks with automatically checked outcomes. Google’s Gemini 3.1 Pro generated the examples as a teacher model. OpenMobile-2, then, is open in part because a closed model helped create its training material.

The release includes two agents: one based on Qwen3.5 with 9 billion parameters, and one based on Qwen3.6 with 27 billion. The researchers describe the models as among the leading current smartphone agents.

For each rebuilt app, the researchers predefine which action changes which state, then verify the result automatically. | Image: Yan et al.

Source: the-decoder.com

More apps, better transfer

Both models improved on established benchmarks. On AndroidWorld, the larger model rose from 67.1 to 79.9, while the smaller one went from 54.3 to 74. OpenMobile-1 scored 64.7 on the same test. On MobileWorld, a harder benchmark with longer tasks spanning multiple apps, the larger model improved from 35.9 to 50.4.

OpenMobile-2-27B hits 79.9 points on AndroidWorld and trails only Gemini 3.1 Pro on SPA-Bench with real devices. | Image: Yan et al.

Source: the-decoder.com

The largest reported gain came on real Android devices. The larger model’s SPA-Bench score rose from 31.9 to 59.6, which the team says is the best result among open models.

More app diversity during training improves performance on unseen apps. After hybrid training, the models solve tasks with tools more often and in fewer steps. | Image: Yan et al.

Source: the-decoder.com

A controlled comparison suggests that training breadth mattered more than training volume. Holding the amount of data constant and increasing only the number of apps used in practice raised success on unfamiliar apps from 60.7% to 67.5%. More examples from a few apps helped less than exposure to more apps.

Reinforcement learning added only a modest improvement, mainly in environments resembling the training set. On AndroidWorld, the larger model’s score after reinforcement learning was actually slightly lower: 79.9 rather than 80.8.

Shortcuts beyond taps and swipes

In an experimental setting, more than 50 simulated apps support direct commands such as “add to cart.” Instead of navigating menus, an agent can use a command when available and switch back to ordinary navigation when it is not.

Without specific training, the models mostly fail to notice these shortcuts. Training changed that: agents solved up to 10.3 percentage points more tasks and used about five fewer steps. The researchers say the skill also transfers to tools the models did not encounter during training.

Late in reinforcement learning, the agent received an extra reward for using a direct command at least once during an attempt. The task still had to be completed to earn the main reward, and agents were penalized for claiming success when they had failed. The interfaces exist only in simulated apps for now; the researchers call the approach experimental.

The team also evaluated its methods on a self-built benchmark of 215 tasks, nearly half involving multiple apps. A program, rather than a language model, checked completion by comparing stored app data before and after each attempt. The authors found little overlap between training and test tasks.

My read is that the strongest result is not the benchmark lead but the training comparison: app variety appears to help agents handle unfamiliar interfaces more than simply adding examples from familiar ones. That is a more useful claim than a general promise of human-like phone use, but it still rests partly on simulated apps and a benchmark built by the team.

A 2025 Microsoft Research study had already framed the trade-off: API-based agents are more reliable, while graphical-interface agents are more flexible, especially where external interfaces are unavailable. Oppo has taken another route with its open X-OmniClaw, which repeats previously demonstrated app routes on real devices through deep links rather than navigating menus. Google has built smartphone control directly into Gemini 3.5 Flash.

What I’d want to know is whether OpenMobile-2’s gains survive when the apps are neither recreated nor equipped with direct commands. The project makes a case for broader practice; the harder test is whether that practice carries over to the messy interfaces people actually have.