An LLM agent that tests its own equations beats symbolic regression baselines

Scientific data often hides simple laws — equations that explain how one quantity depends on another. Finding them is hard: the space of formulas is enormous, measurements are noisy, and brute-force search chokes almost immediately. Symbolic regression is the attempt to recover exactly that kind of compact formula. Most approaches either enumerate expression trees or train a neural network to predict finished formulas. In both cases the LLM usually just suggests ideas, while a human hard-codes the search logic.

The authors of SR-Scientist take a different view: let the LLM be the scientist. The agent analyzes the data itself, writes and runs code, builds and tests equations, optimizes constants, and improves its hypotheses step by step from experimental feedback. It works the way a stubborn researcher works, only at machine speed.

How it works inside

The agent runs a ReAct loop: at every step it reasons, calls a tool, looks at the result, and decides what to do next. There are two tools. The first is a data analyst: it uses code to compute statistics and pull samples. The second is an equation evaluator: it takes the "skeleton" of a formula, fits the constants numerically, and returns metrics. That spares the agent from reinventing the same code forever and gives an honest comparison between competing hypotheses.

The agent has memory — an experience buffer holding the best equations found so far and how good they were. On later iterations the model sees those finds and aims at a tighter error target. That is what sustains progress over a long trajectory, even under a fixed budget of steps and LLM calls.

The metric the search optimizes is MAPE, in percent. It is convenient when the data spans different scales. For comparability with other methods, the authors additionally pick out the formula with the lowest NMSE.

Training with reinforcement learning

To get the agent to do more than think out loud — to drive a hypothesis systematically to a good result — the authors build a synthetic training set spanning four disciplines and train the policy with Group Relative Policy Optimization. The reward is continuous: the closer to the target MAPE, the higher it is. That matters because it thins out the sparsity of the signal: the agent is not waiting for rare wins, it sees a gradient of improvement.

What the experiments show

The LSR-Synth dataset covers materials science, chemistry, biology and physics, with both in-domain and out-of-domain tests. The headline metric is accuracy-to-tolerance: the share of examples whose relative error falls inside a given threshold. On that benchmark SR-Scientist consistently beats the baselines. The absolute accuracy gain runs from 6% to 35% depending on the domain and the model. On GPT-OSS-120B it reaches Acc_0.01 = 63.57% and Acc_0.001 = 49.35%. The contribution of reinforcement learning stands out on its own: on Qwen3-Coder-30B it lifts quality in every discipline.

Numerical accuracy is not the only thing that counts — symbolic accuracy does too, meaning whether the recovered formula matches the true structure. Here the agent is ahead as well: 7–8 exact matches against 4–5 for the best competitors under the same settings. The authors walk through physics cases where the recovered expressions match the reference on the key conditions and on the underlying physics.

What the gains depend on

The ablations are honest about the weak spots. Take away data analysis and quality drops noticeably: the agent has trouble seeing where and why a hypothesis is wrong. Take away the experience buffer and progress falls apart — good structures never carry over from one iteration to the next. One more practical point is the balance between trajectory length and number of iterations: past roughly 25 steps per attempt, the budget is better spent on additional iterations than on ever-longer reasoning.

Why it matters

The large language model here is not a generator of elegant formulas but an active participant in the scientific process. It builds a model of the world on the fly: from data to hypotheses, through code to verification, through feedback to improvement. Minimal hand orchestration, maximal autonomy. That is precisely the design RL training amplifies: the agent learns to think over a longer horizon and act more efficiently.

The takeaway is simple enough: give an LLM the right tools and teach it to live on a long horizon, and it starts working like an attentive researcher. SR-Scientist delivers higher accuracy, better transfer, more robustness to noise and greater symbolic accuracy — all the things that matter in real scientific use.

💾 Code