Mapping code by behavior helps agents plan edits better on fewer tokens

When the problem isn't the model, it's the harness

Conversations about AI agents give almost all their attention to models. Which LLM is stronger, whose reasoning is better, who writes more accurate code. But in real engineering work, an agent's success rarely rests on the model alone. There is another layer that assembles prompts, holds state, calls tools, and keeps the steps in order. The authors of the paper call it the agent harness.

And that layer is exactly where things hurt once the system has to change.

Add a capability. Adjust a behavior. Move logic onto a different API. Fix a rare execution branch. All of it sounds like routine work. In practice, a developer — or a coding agent — first has to work out where in the code the behavior actually lives. And that is usually not one file or one function. The logic is smeared across execution stages, shared state, utility modules, and workarounds.

A new paper, Harness Handbook: Making Evolving Agent Harnesses Readable, Navigable, and Editable, proposes looking at the code not as a set of files but as a set of behaviors. And it shows this is more than a tidy documentation style: it is a working tool that helps agents plan edits better, miss fewer of the places that need changing, and spend fewer tokens.

The real problem

In short, the paper makes a very down-to-earth point: the hardest part of changing a complex system is not writing the new code, it is finding every place that has to change.

The authors give this its own name: behavior localization. It is the search for every region of code that together implements the behavior described in a change request.

Why it is hard:

The usual tools only go so far. Code search, repository indexing, a long context window — all useful, none of them closes the main gap. The agent still has to reconstruct the map itself: which behavior corresponds to which pieces of the implementation.

That gap is what the paper sets out to close.

What Harness Handbook is

The idea behind Harness Handbook is simple and strong: build a behavioral view of the agent harness that links a description of a behavior to the specific code behind it.

Instead of a folder → file → function structure, you get an execution stage → components of that stage → specific regions of code structure. Navigation starts not from the file system but from what the system does at runtime.

The three-level Harness Handbook view: from the whole-system picture down to stages and then to specific regions of code.

The Handbook has three levels:

A state view is kept alongside it: where shared variables and state registers are read and where they are written. That matters, because state is often what stitches together logic living in different modules.

The key principle here is progressive disclosure. First you get the whole picture, then you look at the stage you need, and only then do you drop into the code. It works like a good interactive map rather than an undifferentiated pile of source files.

How the handbook gets built

The interesting part is that the Handbook is not written by hand. It is built automatically from the repository in three steps.

The Harness Handbook build pipeline: static analysis, organization by behavior, and assembly of the three-level handbook.

First comes static analysis. Functions, signatures, call sites, boundaries with external dependencies, and the links between parts of the program are extracted from the code. This runs deterministically, with no model involved.

Then comes the behavioral organization phase. Here an LLM helps work out which execution stage a given function or file belongs to. For small, more legible systems you can work at function granularity. For huge repositories, at file granularity.

Finally the Handbook itself is assembled on top: the L1–L3 levels, the links into code, and the state map.

There is an important safeguard: the Handbook must not hallucinate structure. If a link to a region of code is no longer confirmed by the current repository, that entry is frozen and takes no further part in localization. In other words, the source of truth is always the code itself, not the text description.

That is a sensible engineering trade-off. The authors are not trying to replace the repository with a clever summary. They are building a navigator that gets you to the right places faster.

What this gives a coding agent

Along with the Handbook, the authors propose a workflow they call behavior-guided progressive disclosure. The point is that the agent does not jump straight into code search; it walks down the levels from the top.

The sequence looks like this:

This matters. The Handbook does not edit the code and does not stand in for it. It narrows the search space and helps you not miss scattered edit points.

And once the edits land, the handbook can be resynchronized automatically with the new state of the repository. So it is not a one-time assist; it stays a live map of the project.

How the idea was tested

The authors tested the approach on two open-source agent harnesses of very different scale.

One is a compact Python system with six source files. The other is the large Codex Rust monorepo, with thousands of files and deep call graphs.

For each system they assembled 30 behavior change requests, split into three types:

What was scored was not the final patch but the edit plan: how well the system understood where the code needed to change, whether it inflated the scope of the change, and how well it reasoned. That is a reasonable choice, since the paper is about behavior localization, not the syntax of the final diff.

What came out in practice

The headline result: access to the Handbook improved plan quality and cut token spend at the same time.

With the Handbook, plans win more often and cost the planner fewer tokens on both systems.

On Codex, the plan-quality win rate rose from 28.3% to 38.3%. On Terminus-2, from 26.7% to 45.6%. And this is not one judge's verdict: the finding held up across three different evaluator models.

The token picture is good too. Average spend per request fell:

That is the important part. Quality gains are usually bought with a longer context. Here it went the other way: the right places were found better while less irrelevant code was read.

Another strong result is agreement with reference plans built by more capable models. Given the Handbook, the weaker planner matched the reference answers noticeably better on which files and symbols needed changing. The improvements showed up almost everywhere: recall, precision, and the resulting F1.

The drop in outright misses is the most striking. The Wrong metric is the share of cases where the plan did not overlap with the reference at all. With the Handbook it fell by more than 20 percentage points in places. So the handbook did not just nudge good plans a little higher, it rescued the outright failures.

Where the gain shows up most

The largest gains land on the hard request types: cross-file and search-adversarial tasks.

The clearest improvements came not on the easy cases but on the ones where ordinary code search is weakest:

This fits the paper's central idea. If a change can be found by searching for a function name, the Handbook offers no magic. But when a behavior lives between files, stages, and state, a behavioral map earns its keep.

Why this matters more broadly than it looks

At first glance the work looks narrow: help with editing agent harnesses. But it lands on a much more general class of problem.

There is a lot of talk today about building an agent that writes code on its own. There is less talk about the fact that most real engineering is not generating code from scratch but carefully changing systems that already exist. And there the job is not to invent an elegant function, it is not to forget the hidden dependency and not to break the workaround.

The paper is a reminder of something important: a strong agent needs more than a good model. It needs a good working memory of the system, organized the way a human or an agent thinks about the task. By behavior, not by file.

A few consequences follow:

The idea of rebuilding the Handbook automatically after every diff is the most interesting piece. It looks like the seed of a system where an agent does not just write patches but maintains a current world model of its own code.

The takeaway

Harness Handbook proposes a very practical shift in perspective: see a repository not as a set of files but as a map of what the system does.

The shift pays off concretely. The agent finds edit sites better, misses less often, builds tighter plans, and spends fewer tokens. The biggest gain arrives exactly where ordinary tools fail most: smeared logic, rare paths, and cross-module coupling.

The main conclusion is simple: evolving complex AI systems depends not only on being able to generate a change but on being able to work out where that change belongs. And on that problem, a behavioral handbook may matter as much as the next jump in LLM quality.