An AI agent doesn’t need ready-made answers to get better.

An AI agent can improve more than just its answers—it can improve its own harness

You can give the same language model a task using different tools, prompts, and operating rules—and get different results. The external harness determines how an agent uses tools, stores information, checks files, and divides work among agents.

The authors of the Harness 2 paper propose letting an agent modify its harness on the fly: complete a task, spot a problem, test a new rule, and apply it next time. The model’s parameters stay frozen. Only the setup around it changes.

Evaluating those changes is harder than it might seem. In real-world work, there often isn’t a correct answer or a test program ready to tell you whether things improved. An agent can write more instructions and still make mistakes. And a change that helps with one task can get in the way on another.

Two levels of improvement

Harness 2 is built around two related processes. One improves the harness for the current task. The other preserves experience that could help with future work.

The harness has eight components: a system prompt, constraints, memory, skills, tool-use guidance, scripts, subagents, and extensions. Each serves a different purpose. A skill contains a specialized procedure and loads when needed. A script automates a repetitive action. A subagent handles a separate part of the work in its own context. An extension can monitor tool calls and enforce rules the model shouldn’t overlook.

Eight parts of the harness, from the system prompt and memory to scripts and extensions.

To improve the harness for the current task, Harness 2 follows three steps: propose, test, assemble.

First, the agent reviews the initial task run: the worker’s actions, the tools it used, and the result. It isn’t shown the correct answer. The system then prepares two sets of changes. One targets general problems—for example, poor planning or failure to check a completed file. The other reflects the needs of a specific domain.

Both sets are tested on the same task. The system compares not only the output files but also how the agent worked. If a change prompts the agent to use a needed tool or fix an error, that’s useful evidence. But it doesn’t prove the result is correct: both runs may make the same mistake, and the model’s behavior can vary from one attempt to the next.

Finally, the system selects which changes to include in the new harness. It can combine useful parts of both sets, take one as is, or leave the harness unchanged. The agent then runs with the resulting harness and produces an answer to the current task.

The system first tests harness changes on the current task, then saves useful experience for future tasks.

Remembering how to fix things

After the work is done, Harness 2 updates its experience library. It stores the accepted harness along with an explanation: what problem was noticed, what change was tested, and what observations supported it. The library doesn’t become a collection of commands that the system automatically copies into every new task.

An instruction that helps with one job may be wrong for another. So the system starts each time with the same general-purpose base harness and uses the library as a source of examples and ideas. Components can serve as a starting point, but their facts and rules need to be checked against the new task.

The authors tested two ways to broaden the search for improvements. Parallel mode creates several versions at once, which can be compared and combined into a new harness. Sequential mode makes changes step by step, with each step taking the previous results into account.

In both modes, the number of attempts sets the search budget. More attempts mean more computation and more chances to find a useful change. In parallel mode, versions can run simultaneously if resources allow.

Parallel mode tests several versions, while sequential mode develops the harness one step at a time.

Test results

The researchers evaluated the system on three benchmarks: WorkBuddy-Bench, which covers coding, web, and office tasks; JobBench, for long-running professional tasks; and the Harvey benchmark for legal work. They used Gemini 3.5 Flash and GLM-5 with two base harnesses. The models themselves were not fine-tuned during the experiments. The main results were averaged across three runs with different task orders.

A single improvement cycle boosted both models across all three benchmarks and every model-and-harness combination tested. On WorkBuddy-Bench, GLM-5 improved by as much as 11.5 points. The maximum gain on JobBench was 4.5 points, while on the legal benchmark, the score for criteria met rose by 11.8 percentage points.

Additional cycles brought further gains. With a parallel search budget of five versions, results improved by another 2.9–3.9 points on WorkBuddy-Bench and 3.0–4.2 points on JobBench. On legal tasks, the additional gain was 2.6–6.7 percentage points.

Results improve as the search budget grows; in this example, Harness 2 beats the base harness across every WorkBuddy-Bench domain for Gemini 3.5 Flash.

The gains are most noticeable where the base model is weaker. On some WorkBuddy-Bench tasks, GLM-5 with an improved harness outperformed Gemini 3.5 Flash with its base harness. Results depend not only on a model’s capabilities but also on how it uses tools and organizes its work.

The authors also tested whether accumulated experience could transfer to other tasks. A library built on JobBench produced a small improvement on WorkBuddy-Bench. And a harness developed for Gemini 3.5 Flash improved other models’ results on some WorkBuddy-Bench tasks. The effect varied by model and domain: experience can transfer, but it won’t help equally in every case.

Removing individual components also hurt performance. When the researchers disabled the library, the separate handling of general and domain-specific problems, or the process for combining changes from different versions, scores fell. Improvements depend on testing changes, combining them effectively, and drawing on accumulated experience—not just on one good instruction.

Where the method can go wrong

Harness 2 has no direct way to know whether the work is correct. It relies on differences between runs and model-based assessments. This can reveal how the agent’s behavior changed, but it doesn’t replace checking the result. If two runs rely on the same mistaken assumption, comparing them may not expose it.

The authors describe cases where the system chose a weaker answer. In other examples, an overly narrow change made it harder to account for all the necessary conditions. Sometimes it was difficult to separate the effect of the harness from ordinary variation between runs or quirks in the evaluation process.

More versions mean more runs and higher inference costs. In the experiments, additional computation improved results, but that doesn’t mean the maximum budget is always the right choice. The system still needs to learn when it’s worth continuing the search and when it has enough evidence.

Conclusion

Harness 2 can improve an AI agent’s harness for the current task and preserve tested experience for future work. This can raise performance without fine-tuning the model and help carry successful approaches across similar tasks.

Comparing behavior is only an indirect check. It shows that the agent acted differently, but doesn’t prove the result is correct. Systems like this can speed up harness improvements, but they still need reliable quality checks and controls on the cost of additional runs.