A method presented at ACL 2026 builds tool-use training data backwards: assemble a working chain of API calls first, then write the user request it answers. The approach, called ToolGrad, produced a 500-example dataset that was enough to fine-tune a 12B Gemma-3 to 83.1 on the Berkeley Function Calling Leaderboard — against 83.2 for gemini-2.5-pro, 82.8 for claude-4.5 Opus and 74.4 for gpt-5. The benchmark runs on a different set of APIs from the ones the training data was built from.
Source: research.google
The problem it addresses is mundane and expensive. Agents that search the web, read local files and run generated Python need models that call tools correctly and efficiently, and training those models requires datasets pairing user requests with the tool-call chains that satisfy them. The authors' earlier work labelled such data by hand, which does not scale to anything a modern model would need.
ToolBench and ToolACE automated the labelling with an agent that hunts for a solution by trial and error: pick a group of APIs from a library, invent a plausible user instruction over them, then send a depth-first search agent to find a chain of calls that satisfies it. The inefficiency is structural. Useful action sequences have to be mined out of a sprawling exploration, and because the request is written before anyone knows whether it can be served, a large share of attempts fail validation.
ToolGrad inverts the order. It builds a verified chain of calls first and generates the matching user prompt afterwards. The argument for why this is easier is the interesting part: a finished solution carries more unambiguous information than the request that prompted it, so going from chain to prompt is a single LLM step rather than a search. Every example is answerable by construction, and the authors report a validation success rate approaching 100 percent.
The "gradients" in the name are borrowed. Conventional machine learning computes numeric gradients of error over minibatches and hands them to an optimiser that updates weights. TextGrad moved the same structure into prompt optimisation, with an AI critic producing detailed feedback in ordinary prose; those textual gradients steer the rewrite of the prompt. ToolGrad applies the idea to synthetic data generation, where the thing being optimised is not a static string but a growing scenario of API calls drawn from a large tool library.
Four modules run the loop. The API Proposer narrows a randomly drawn set of APIs down to a few candidates that could plausibly extend the current scenario. The API Executors test those candidates in parallel and produce detailed execution reports. The API Selector reads the reports and picks the single most successful call — this is the textual gradient, setting the direction of the next step and appending the chosen call to the chain. The LLM Updater then rewrites the synthetic user request and the AI response so both match the enlarged set of APIs. Repeat, and the output is a user request, a validated chain and a final answer.
Source: research.google
For the cost experiment the authors used ToolBench as the API library, which holds more than 16,000 real APIs, and compared the original query-first depth-first search against their answer-first loop. ToolGrad generated more complex data, raised the share of examples that passed validation and lowered generation cost.
Source: research.google
The benchmark run used small ToolGrad-500 datasets built from ToolBench APIs to fine-tune Gemma-3 at 1B, 4B and 12B, yielding ToolGrad-1B, ToolGrad-4B and ToolGrad-12B. Evaluation was on the Berkeley Function Calling Leaderboard, which uses APIs that do not overlap with ToolBench, against three comparison groups: the base models without fine-tuning, proprietary frontier models from Gemini, GPT and Claude, and specialist tool-use models ToolACE and Hammer-2.1-7B.
Fine-tuning improved tool-use scores at every model size. ToolGrad-12B reached 83.1, within a tenth of a point of gemini-2.5-pro at 83.2 and above claude-4.5 Opus at 82.8 and gpt-5 at 74.4. The ToolGrad-500 data was itself generated using gemini-2.5-flash-lite, and the fine-tuned Gemma-3-12B scored above that teacher model. ToolGrad-12B also beat ToolACE, which was trained on a more advanced API library.
Source: research.google
Two of those numbers deserve a second look. A benchmark on which gpt-5 lands nearly nine points below a fine-tuned 12B model is not measuring reasoning; it is measuring conformity to a calling convention, and conformity is the cheapest thing supervised fine-tuning buys. That does not make ToolGrad's result uninteresting — it makes the 83.1 a statement about how much of tool use is protocol rather than intelligence.
The student-beats-teacher finding is likewise less mysterious than it sounds. The information that lifts Gemma-3-12B above gemini-2.5-flash-lite does not come from gemini-2.5-flash-lite. It comes from the API Executors, which call real endpoints and report what actually happened. The teacher proposes; the world selects. Framed that way, this is grounding rather than self-improvement, and grounding is the part that stops working the moment the APIs go quiet.
The question the paper's framing leaves open is what the near-perfect validation rate actually certifies. Because the user request is reverse-engineered from a chain that already ran, every example is guaranteed solvable — which is precisely why validation almost never fails. Nothing in the loop asks whether the request is one a person would ever make. A dataset of perfectly solvable synthetic questions trains models that are excellent on tasks shaped by the structure of an API graph and untested on tasks shaped by human intent, and the reported cost advantage is stated as a direction rather than a multiple.
The bulk of the research was done by Zhongyi Zhou during a stint as a visiting researcher at Google. The authors credit contributions from Kohei Uehara, Haoyu Zhang, Jingtao Zhou, Lin Gu, Zheng Xu and Tatsuya Harada, and strategic guidance and review from Adarsh Kowdle and Shahram Izadi. Planned extensions cover larger and more dynamic API ecosystems, and carrying the self-improvement property into continuous on-the-fly learning for personalisation over time.
If 500 synthetic examples put a 12B open model within a tenth of a point of the best proprietary system on a tool-use leaderboard, the leaderboard and the capability have come apart. The part of agent-building that everyone currently measures is now the cheap part.