A language model can already run a robot: understand a request, break it into steps and call the right skills. But in the physical world, reasoning correctly is not enough. A decision has to arrive before the situation changes, and success has to be proven by evidence, not by a skill reporting “done”.
The core idea
The field calls this agentic robotics. Instead of training a separate policy for every family of tasks, a general model runs the robot in a closed loop: it watches the world, works out what the person wants, reasons, calls skills, checks the result and changes the plan. The trigger was the release of GPT-6 Astra: within weeks, models with agentic harnesses matched or beat purpose-trained policies on many embodied benchmarks. The paper answers three questions: what such a robot is made of, how to test it fairly, and what still holds it back.
What an agentic robot is made of
The reference design splits the robot into three systems. Above them is the person and their intent, below them the body and the physical world, and off to the side sits an event monitor that is not part of the command chain.
Anatomy of an agentic robot: S2 — a model plus a five-part harness — makes the decisions; S1 skills and S0 body control run at their own rates, and the event monitor can interrupt S2 or slow the robot at once.
Commands flow down, evidence flows up. The split is functional: one model can play several roles, and S2 can send a target straight to S0 or write code that becomes a skill itself.
How it works: the robot looks for a red cup
The request: “Find the red cup and bring it to the dining table.” S2 breaks it into four subtasks — locate, grasp, carry, place — and runs each through a six-step agentic loop.
The agentic loop: six steps from intent to a verified outcome; the event monitor can interrupt it at any step.
First S2 calls perception, and the harness records where the cup is and when it was seen. Then it calls the grasp skill and uses a fresh observation to check that the cup is held securely. Once the robot has put it down, S2 confirms that the cup is standing on the table where it was asked to be. Along the way, the harness keeps a past event, “the cup was grasped”, apart from a current state, “the cup is still held”.
The interesting part starts when something goes wrong.
That is why verification is four separate questions: was the request accepted, has execution ended, is there evidence that the intended physical effect holds, and does that effect satisfy what the person wanted.
How do you test a robot that thinks?
Two months after Astra came out, results had become hard to compare: on RoboDojo alone, reports give it anywhere from 22% to 85% success on the same 42 tasks, depending on the harness and the protocol. So evaluation is split into two levels. The model is tested on its own: does it know, understand and reason — in a single answer, with no tools. And the whole system is tested separately, in a closed loop, with its harness, skills and body.
What to measure in a robot agent: nine dimensions and 53 sub-dimensions; 29 of them depend on the body, and standard tests of language models rarely cover them.
The result is a profile across nine dimensions, not a single score: a success rate cannot tell you how safe, fast or expensive a robot is. To make numbers from different groups comparable, the paper proposes rules:
Key numbers
The first two numbers are about time. When the world kept moving, robots completed the task almost as often, but collided 12.3 times more on the way: on the hardest routes Astra spent 7.7 seconds per decision, and by then the situation had changed. Hence the paper's central principle: not “think longer” but get smarter within the deadline — leave fast reactions to the skills and the body, and give the model only what cannot be solved faster. The third number explains why other groups' percentages cannot be set side by side without describing the harness. The fourth is about safety: refusing in a chat does not guarantee refusing in action, and that has to be tested where the robot actually moves.
What it means
On narrow simulated suites, a general model with a harness already competes with trained policies and sometimes beats them — though at times with hints the policies never got. On broad tasks that require exploring the environment, success is still low: 19% on RobotWorld, 23.2% on RoboQuest, 16.8% on HumanCLAW. On the last one, the best model found the target in 64.9% of episodes but finished the task in only 16.8%: finding an object is not the same as dealing with it. The weak spots are physical and evidential: precise manipulation, locomotion (Astra solved none of the 11 locomotion tasks in RobotWorld), understanding its own body, and admitting that an answer cannot be known.
The ladder of delegable work: progress is measured by the responsibility a robot can be trusted to carry alone.
So the paper proposes to measure progress not by the number of solved tasks but by the responsibility a robot can be trusted with. Robots already execute single actions and complete whole tasks, they are only beginning to reach workflows of several tasks, and no system in an everyday, unprepared environment has yet been reported to hold a standing role — say, running a household's daily breakfast — without human help. To climb higher, a robot needs more than intelligence: it has to be on time, stop when told, and prove that its action really changed the world the way it was asked to.
Read next
WorldFoundry: How can you distinguish video generation from world
How AI Can Enhance an Individual’s Intelligence
FrontierPhysics: Let’s see whether AI agents can make discoveries
How AI Chooses the Right Model for Each Task
CaPtain: How to control a robot using a small AI model
WorkFactory: Hundreds of thousands of professional tasks helped train
How AI Agents Learn General Skills Across Different Tasks
How can you tell that an AI agent is doing the task incorrectly?
How an AI agent can master a new skill through practice using the Socratic method
How an AI agent improves interfaces on its own
How AI Predicts Brain Activity
An AI agent doesn’t need ready-made answers to get better.
AI reviews in simple way
Every day we read fresh AI papers and retell the essentials in plain human language — no hype, no fluff. If you want to see where AI agents are heading before everyone else, subscribe.
New reviews — every day
Follow on X