AllSpark has published two open-weight search agents, Iris-mini at 35 billion parameters and Iris-pro at 397 billion, and alongside the scores an argument that search-agent benchmarks measure software as much as they measure models. The team ran every evaluation twice, once with context management and once without. On BrowseComp the gap reaches 21.2 points for the smaller model. That is wider than the margin separating most of the systems these numbers are used to rank against each other.
Both agents are built on the Qwen series: Iris-mini on Qwen3.6-35B-A3B, Iris-pro on Qwen3.5-397B-A17B, each with a 256,000-token context window. The team says both are the strongest open-weight search agents in their respective size classes.
The evaluation covers four benchmarks: BrowseComp, which asks for rare facts reachable only through indirect clues; BrowseComp-ZH, its Chinese counterpart; DeepSearchQA, which grades the completeness of the evidence an agent gathers; and Humanity's Last Exam, expert-level academic questions. With context management on, Iris-mini scored 82.2, 84.8, 86.9 and 52.3. Iris-pro scored 88.6, 85.1, 92.9 and 56.4.
Iris-mini takes first place in its size class on three of four benchmarks, losing only to XYZ-Aquila-mini on DeepSearchQA
Source: the-decoder.com
In the smaller class Iris-mini leads on three of the four. On BrowseComp it beats the next model, XYZ-Aquila-mini, by 3.4 points, and loses to it on DeepSearchQA. In the larger class Iris-pro leads or ties, and on some tests comes close to systems that need considerably more compute, the authors say. All Iris figures come from a single agent: no helper agents, no extra verification stage at the end. That constraint is the sort of thing usually mentioned in a footnote, if at all, and it makes the numbers harder to compare upward and easier to trust.
The context-management finding is the part of this release worth arguing about. The team's claim is that how an agent's context is managed during a long run sometimes moves the score more than the stated difference between two systems. During extended research the window can fill before the agent has even worked through all its sub-questions; tricks like deleting conversation history artificially extend the run and say little about the model underneath. So AllSpark evaluates each benchmark both ways, holding tools, context limits and judge model constant.
The smaller model gains the most from it, and the reason given is not a smaller token budget but a faster burn rate: Iris-mini takes more steps to solve the same tasks and hits the window limit more often. On Humanity's Last Exam the gain is smaller, because that benchmark leans on subject knowledge and academic reasoning, with web search in a supporting role. The best configuration combines history deletion with a second attempt: when the first run fails, the system compresses it into a short note recording which options were already checked and which directions were ruled out, then appends the note to the task for the next run.
The training data is built backwards from the link structure of web pages. The pipeline takes a source page and the links leading out of it, builds a graph of terms and relations, and from that graph composes a multi-hop question whose answer requires traversing several connected steps. Every term except the final answer is then replaced with a paraphrase, so the question cannot be solved by text matching alone. Only questions a reference model fails without tools but solves with the right sources make it into the set, which keeps them hard and verifiable at the same time.
From there a stronger teacher model generates solution chains with reasoning, queries and results. Each chain is checked twice: once as a whole, for correctness, repeated loops and search depth, then step by step by a judge model whose criteria, the authors say, were derived from the data rather than written by hand. Reinforcement learning against the live internet follows, with the judge model and the result summaries running inside the training cluster on the team's own large Qwen model, so the process does not depend on outside services. Supervised fine-tuning and reinforcement learning then alternate in what the authors call SFT-RL climbing, with the hardest solved tasks and the most efficient solution paths from each round carried into the next.
What makes this release interesting is not that an open model scored 88.6 on something. It is that AllSpark has effectively told everyone the scoreboard is unreliable and then published the instrument for checking it. If a 21.2-point swing can come from context handling alone, then a leaderboard reporting only managed-context runs is reporting a joint result of model and harness without saying so. That is a strong claim to make while also claiming to be in first place, and the honest version of it cuts both ways: some portion of Iris's own margin belongs to the harness too.
The appendix contains a second quiet indictment of the benchmarks. On a BrowseComp-ZH question about Game of Thrones, the agent answered Bolton and was marked wrong against a reference answer of Lannister. Sansa Stark's second marriage is to Ramsay Bolton, so the agent was right and the answer key was not. The team frames this as evidence that benchmarks need improving. It also means the reported ceilings on these tests are soft in a direction nobody accounts for.
The finding I would want independently confirmed is the side effect. The team reports that its data and specialised models improved performance on tasks they were never trained for, including general tool use and office work, and suggests search may be a base skill rather than a narrow specialisation — that what an agent learns while working with incomplete information transfers to everywhere else it has to work with incomplete information. That is the most consequential claim in the paper and the one supported by the least detail here.
Weights for both models are on Hugging Face and the code is on GitHub. The release includes Iris Harness: the agent loop, the tools, the context-management strategies and the evaluation across all four benchmarks, working against any OpenAI-compatible endpoint. The data and training pipelines are promised later. So AllSpark has open-sourced the measuring instrument and held back the factory — anyone can now audit the context-management effect on their own systems, and nobody can yet build an Iris.