GPT-6 Astra scored 62.7% on ARC-AGI-3, the benchmark that drops a model into unfamiliar game worlds without explaining the rules or the goal and makes it work them out by trial and error. GPT-5.6 Sol, its predecessor, scored 7.78%. Claude Opus 5 scored 30.16%. About six months ago, after ARC-AGI-3 shipped, François Chollet put the benchmark's saturation roughly a year out, depending on how deliberately labs optimized for it; Astra arrived in about half that time. Asked whether his 2030 forecast for AGI still stands, Chollet said the date should move earlier — progress is running faster than he expected.
That is the headline result. The rest of the benchmark record for Astra is considerably less tidy, and two independent labs that aggregate dozens of individual tests into a single score reached opposite conclusions about the same model.
Epoch AI, pulling from more than 50 benchmarks, gave Astra 169 points and first place among 267 models. Artificial Analysis, testing knowledge, coding and text comprehension, gave it 61 — level with its own predecessor and behind Claude Fable 5.1 at 66. When a composite score can rank a model first out of 267 or behind a competitor and level with the thing it replaces, depending on which tests go into the basket, the composite has stopped being a measurement and become a weighting choice.
The pricing picture splits the same way. OpenAI charges 2.5 times more per unit of processed text for Astra than for Sol, which works out to roughly 75% more per completed task. Against Anthropic the comparison inverts: on coding tasks Astra matches Claude Fable 5 while costing less than half as much per task, because it is frugal with compute — a third of Sol's reasoning steps and a fifth of the volume Opus 5 requires. On the coding agents index Astra scored 67 using about a third of Sol's tokens; Fable 5.1 leads at 70. Hallucinations in AA-Omniscience dropped from 92% to 51%. Against that, Astra lost about 80 Elo in GDPval-AA v2 and came out weaker on banking customer support, SciCode and long-context reasoning. A footnote to the results notes that GPT-6 Astra was tested at a very high reasoning level, and that at maximum it reaches 97.5%. ARC-AGI-1 is now treated as a largely saturated benchmark.
On Epoch AI's individual measures, Astra leads in math, knowledge and puzzles while Fable 5.1 takes nearly every coding test — but the two have not been compared on equal footing. Epoch AI has exactly one coding result recorded for Astra, from a run at medium reasoning. Any statement about which model codes better is currently resting on that.
In math the standout is FrontierMath Erdős, where Epoch AI reports Astra as the only model to have solved two of 68 open Erdős problems with proofs verified in Lean, at up to $300 per attempt. Three further solutions came from additional non-standardized runs that consumed more than $220,000 in compute, which Epoch AI excludes from the official count. Excluding them is the right call and also the interesting one: the gap between $300 and $220,000 is the gap between a capability and a demonstration.
Back on ARC-AGI-3, OpenAI's own claim was 99.9%, obtained under different conditions. Astra ran inside a test harness OpenAI built, which preserves reasoning chains between separate requests and automatically compresses long runs. ARC Prize measured those runs as roughly 3.66 times faster and 49% cheaper in tokens than runs on its internal harness, comparing across 167 pairs of game tasks both setups solved. ARC Prize insists that only the 62.7% figure, produced on its own scaffolding, permits fair comparison between models from different vendors; it plans to publish vendor-harness numbers separately in future. Harnesses and the score inflation they produce were already a point of friction between ARC Prize and OpenAI during the GPT-5.6 Sol evaluation.
One finding runs against the grain of everything else in AI economics. Turning reasoning up made the ARC-AGI-3 run cheaper, not more expensive: on ARC's standard harness the cost falls from $49,791 with no reasoning to $26,098 at maximum, while the score climbs from 35.2% to 62.7%. ARC Prize's explanation is that Astra solves games in fewer moves, so it makes fewer calls and burns fewer tokens. There is an unexplained hole in the middle of that curve: the "low" setting produced 17.5%, worse than running with no reasoning at all, and ARC Prize offers no account of it. Astra has previously solved long-action-chain tasks without separate reasoning, so the new architecture may be doing part of the work before the first token is generated.
Efficiency, not the final score, is what ARC Prize says it cares about. Before launch it had roughly 500 testers play with no pre-screening and recorded, for each level, the median number of moves among those who solved it. On OpenAI's harness, Astra cleared 96% of levels in fewer moves than the human median, averaging a little over half the human move count. That metric is not tracking compute. It is tracking how much interaction with an environment a system needs before it has understood it — precisely the axis on which the organizers expected humans to hold a durable lead. For brute-force approaches the lead holds. For frontier models, ARC Prize describes an almost binary picture: once the model grasps the mechanics, its execution lands inside the human efficiency range.
The mechanism behind that is the part worth reading twice. To solve the games, Astra keeps its own notes and invents a compact, algebra-like notation for objects, coordinates, rules and unfinished plans — `extend8 to3; retract10 to2` for a move sequence, `Turn 5: P=(24,20), empty, facing west` for a game state. ARC Prize has seen similar behavior in other models but singled out the precision and information density of Astra's notes. On the standard harness it matters more than it sounds: anything not written into the visible notes is effectively lost to the model. Chollet described this as effective symbolic world modeling on the fly, per game and per level, with the model building its own shorthand language for describing situations — in effect a bespoke algebraic notation for each game. What matters to him is the provenance: this kind of symbolic modeling used to appear only when a sophisticated external harness supplied it. He reads the shift as harness capability migrating into the model itself.
A third setup tested what Astra does with tools. PRO-LONG is an agent environment built by an outside team, which ARC Prize deployed in advance as a partner for probing ARC-AGI-3 for vulnerabilities and mapping the benchmark's limits. There the model got a sandbox and could execute its own code. Astra wrote small per-game libraries:
board parsers;
state models;
search algorithms;
planners.
In a maze game with guards it built, in sequence, a pathfinder, a combat-rules module, a model of patrol movement, and a program that continuously checked its own predictions against observations. ARC Prize saw no attempts to escape the sandbox. These numbers are not comparable to human performance — the testers had neither a code interpreter nor a notepad — and what they measure is the model working together with tools it wrote.
My reading is that the harness fight is the story, not a footnote to it. OpenAI's 99.9% and ARC Prize's 62.7% are the same model on the same benchmark, separated by a piece of software one of the two parties wrote. ARC Prize is right that cross-vendor comparison requires its own scaffolding, and right that Fable 5 and Fable 5.1 have not run the benchmark at all, which leaves the Anthropic comparison genuinely open. But the harness gap is not noise to be subtracted. Chollet's own argument — that scaffolding capability is moving into the models — implies the boundary between model and harness is exactly what is changing, and ARC Prize is measuring across a line that is dissolving. Meanwhile the "reasoning gets cheaper" inversion and the unexplained 17.5% dip are the sort of anomalies that get investigated only when the headline number is disappointing. This one is not, so nobody will chase them.
The cost comparison deserves the same skepticism. Human testers were paid $115 for a 90-minute session plus $5 per solved game, which at about nine attempts comes to roughly $12.78 per game — money that mostly buys someone's time and willingness to show up. Counting only the electricity equivalent of the brain's metabolic work, ARC Prize puts the human cost at 0.067 cents per game. ARC Prize acknowledges what that number leaves out. It is also the number that will travel, against Astra's $26,098, and it excludes everything that makes a human available to play in the first place.
ARC Prize does not present any of this as evidence of general intelligence, and Chollet is explicit that what is known about the system is how it performs on benchmarks. At ARC-AGI-3's launch the organizers said the same: solving it does not demonstrate AGI, and the benchmark was never meant as a finish line. It tests qualities expected of a general system — exploration under uncertainty, adaptation without instruction, causal world modeling from small amounts of data — but at small scale. Games are completed in spans orders of magnitude shorter than real tasks, which means less data, less modeling complexity and less on-the-fly learning than the real thing demands.
ARC-AGI-4 is already in development. Work began after ARC-AGI-3 shipped, publication is planned for the first quarter of 2027, and ARC Prize is candid that ARC-AGI-3 is a limited test: deterministic mechanics, closed goals, no representation of an open real world. The next generation is meant to probe recursive self-improvement and open-ended innovation. That leaves ARC Prize in an uncomfortable position for the next two quarters — holding a benchmark whose most important result arrived twice as fast as its own co-founder predicted, and whose successor was designed against a gap that has already moved.