Planning a repository as a graph beats Claude Code by 27 coverage points

Large language models write functions and individual files with confidence, then lose the thread when they have to assemble a whole project. Over a long horizon natural language stops being reliable: vague phrasing, mismatched interfaces, leaking dependencies, structure that falls apart. The agent changes its mind mid-task, tests drift, and the codebase turns into a pile of fragments.

The idea: Repository Planning Graph

The researchers propose replacing the vague plan with a durable graph. The Repository Planning Graph (RPG) is a single representation in which nodes hold files, classes and functions, while edges record semantic relationships, data flows and implementation order. Two worlds meet in one format: what is being built, and how exactly it will be laid out in code.

How ZeroRepo works

RPG is not just an idea but a working framework, ZeroRepo. It assembles a repository in three steps:

1) Capability-level planning: a short specification selects the relevant subtree out of a huge feature tree (over 1.5M nodes). Reorganization happens here too — related functions are grouped into modules with good cohesion.

2) Implementation planning: the graph is extended with a file structure, interfaces and typed data flows. A topological order emerges — what to write, and in what sequence.

3) Graph-guided code generation: node by node, with test validation in TDD style. Only what passes the tests lands in the repository.

How they evaluated it

To measure quality, the authors built RepoCraft — a benchmark of six real Python projects (counterparts of scikit-learn, pandas, sympy, statsmodels, requests, django) and 1,052 tasks. The requirement is strict: from a minimal description, build the entire repository and pass the reference tests. They measure functional breadth (coverage), correctness (pass rate), code scale and novelty.

What it looks like in practice

On RepoCraft, ZeroRepo opens a clear lead. It generates about 36,000 lines of code on average — roughly 3.9× more than the strong Claude Code baseline, and tens of times more than the rest of the field. Functional coverage reaches 81.5% and the share of passing tests is 69.7%, which is 27.3 and 35.8 percentage points above Claude Code respectively. It also matters that RPG stabilizes interfaces, helps hold module boundaries and keeps data flows consistent.

What the analysis adds

• Near-linear scaling. Both feature count and lines of code grow close to linearly as long as there is budget for planning. Language-only pipelines without a graph hit a plateau quickly.
• The agent's grasp of the repository. Localization logs show RPG cuts search and edit steps by 30–50%: there is a place to look, a set of dependencies to touch, and a clear picture of what a change will break.
• Controlled novelty. The system does not merely copy known components; it proposes new functionality (11–13% novelty) without tearing the architecture apart.
• Different models, different trajectories. Qwen3-Coder expands coverage more aggressively; o3-mini is more restrained and spreads features more evenly across subgraphs. The two complement each other along the completeness/precision axis.

Why it matters

RPG turns a vague plan into a formal artifact that an LLM and an engineer read the same way. It buys predictability: topological order, stable interfaces, explicit data channels. In that setting it is easier to scale development, bring in new agents and new people, and do targeted debugging and integration.

But RPG is no magic wand: the system leans heavily on the quality of the input specifications, the tests, and how well the global feature tree covers domain scenarios. Complex ecosystems will need precise domain tuning and careful handling of the project's infrastructure pieces. Still, the direction looks mature: graph representations remove the main barrier to long-horizon planning.