OpenAI's GPT-6 Astra fully completed 7 out of 100 tasks on StationeryBench, a new robotics benchmark built around five desk-level manipulations: pulling the cap off a marker, pouring out paper clips, handing a ruler from one robot arm to the other. Ai2's MolmoAct2, driving identical bimanual YAM robots across the same run of 200 trials, completed none. The median progress score was 46 out of 100 for Astra and 12 for MolmoAct2. Results, videos and source code are published on GitHub.
Source: the-decoder.com
Two numbers, two different stories. Seven versus zero is the one that gets quoted, and it is close to meaningless in isolation — both models fail the overwhelming majority of the time on tasks a child finishes without looking. The median progress score is the more informative comparison: 46 against 12 says Astra is not merely luckier at the finish line, it is getting substantially further into each attempt before something goes wrong. A model that reaches the halfway mark and drops the ruler is a different kind of system from one that never picks it up.
Yoav Artzi, an AI researcher at Cornell and Google DeepMind, called Astra a sharp jump in spatial reasoning. On REMAP, a benchmark that has not been published yet, he says GPT-6 Astra scores close to human-level accuracy — while noting that even Astra falls short of human performance in other scenarios. His hypothesis for where the gain came from: OpenAI trained on large volumes of 3D data, Blender scenes among them. That would fit the shape of the results, since Astra's improvements are concentrated precisely in 3D tasks.
If the Blender hypothesis holds, it is the most consequential detail here, and it cuts both ways. Rendered scenes are the cheapest 3D supervision in existence — effectively unlimited, perfectly labeled, generated without a single robot arm. That is an enormous advantage for the company that industrializes it first, and OpenAI has said its long-term plans include building its own consumer robots. It also explains the gap between the two benchmarks. Near-human accuracy on a reasoning evaluation and a 7 percent completion rate on a table with real objects on it are not contradictory results; they are what you would expect from a model that has learned to represent space beautifully and has barely learned to act in it. Rendering teaches geometry. It does not teach friction, compliance, or what a marker cap does when your grip is two millimeters off.
That is also where the reporting outruns the evidence. The strongest claim in the story — near-human spatial reasoning — rests on a benchmark nobody outside can inspect, while the weakest result comes from the one anybody can download and rerun. That asymmetry is worth holding onto. StationeryBench deserves credit for the opposite habit: publishing the videos and the code alongside the score, which is more than most robotics demonstrations come with, and the reason we can argue about the 46 at all rather than watching a montage.
The thing the comparison doesn't carry is how partial credit is assigned. A median of 46 could mean the robot consistently gets to the last step and fumbles it, or it could mean the scoring generously rewards approaching the object. Those imply very different distances to a working product, and the headline number cannot distinguish them.
For now the honest summary is that OpenAI's model is the first in this comparison to be visibly bad rather than absent. That is real progress on a benchmark designed to be hard, and it is roughly 93 completions short of a robot anyone would put in a kitchen.