Finding the causal step fixed three times as many failed agent runs

When an AI agent fails, the guilty step is already behind you

There is an uncomfortable truth about AI agents: the failure almost never happens where you see it. The final answer can be wrong because 20 steps earlier the agent missed a constraint in the task, pulled the wrong record out of memory, or handed off to another agent without the context that mattered. The logs show you the symptom. The cause sits further back.

That gap is what AgentDebugX aims at — an open toolkit for debugging AI agents from researchers at the University of Illinois, Toronto, Google and Stanford. The idea is simple: don't just display the execution trace, close the whole loop — find the failure, work out its cause, propose a fix, and immediately re-run the task with that fix applied.

For today's AI agents this is an intensely practical thing. The longer the chain of actions — and the more tools, memory, browser use, external APIs and multi-agent coordination there is — the harder it gets to tell where exactly things went wrong. Observability without diagnosis does not get you far here.

The overall AgentDebugX design: a closed loop of finding the failure, finding the cause, proposing a fix and re-running.

What AgentDebugX is

AgentDebugX is built as a closed loop of four steps:

This is an important difference from ordinary observability tools. Those are usually good at collecting traces neatly: LLM calls, tool invocations, handoffs between agents, UI actions, search results. But past that point the developer is left alone with the questions: who made the mistake, why that particular mistake was fatal, and how to repair it now.

Here the authors are trying to turn debugging from log-reading into a workflow.

Internally the system has a single portable trace format. That is not a small detail either. Today one agent lives in one framework, another in a different one, and a third just writes raw events. If you want to compare failures, store them, come back to them and run them through different diagnostic methods, you need a common format. That is exactly what AgentDebugX offers.

One more detail: the diagnosis is not stitched into the original trace. The execution history stays as it is, and different versions of the analysis can be layered on top of it. For engineering work that is convenient: you can re-check contested cases, compare methods and assemble regression tests.

Why the root cause is so hard to catch

The authors state the core problem precisely. In AI agents, the visible failure and the causal step often do not coincide.

A simple example: the agent gives a wrong answer at the end. But that final error may have started earlier:

Because of this, reading the whole trace does not always help. Read it once from start to finish and a model — or a person — tends to latch onto the loudest symptom near the bottom. A failed tool call, for instance. But the cause may not be in the tool call at all; it may be that two blocks earlier a wrong intermediate conclusion was accepted.

Hence the paper's main technical move: diagnosis should not be single-pass, it should be multi-round.

How DeepDebug works

The heart of the system is called DeepDebug. It is a separate diagnostic agent that does not merely read the trace but runs a small investigation over several passes.

The logic goes like this.

First DeepDebug reads the entire trajectory and puts forward an initial hypothesis: which step was the decisive error. This helps, because the global pass keeps the goal of the task and the overall context in view.

Then comes a second pass, this one more structural. If the system is multi-agent, DeepDebug walks the chain of handoffs backwards: who passed what to whom, where the information got distorted, at which transition the task was effectively doomed. If there is a single agent, it applies something like a binary search over steps: the diagnostician narrows the search window bit by bit so as not to lose focus.

Then the interesting part. If the two hypotheses agree, the system accepts them. If they do not, a cross-check kicks in: DeepDebug compares the two candidates side by side and picks the one whose fix would most likely have prevented the failure.

What comes out is not just a label reading "error at step 17" but a structured report:

That is a good engineering format. It works for a human and for an automated re-run loop alike.

What it looks like in practice

AgentDebugX has a command-line interface and a web interface. The flow will look familiar to anyone who has ever picked apart a failed agent run: open the trace, move from the visible symptom to the real cause, read the diagnosis, create a new execution branch and compare the result against the original.

The AgentDebugX interface: from picking a failed run to jumping to the causal step and creating a new branch with the fix.

Notably, the authors keep the system "local by default". All artifacts stay on your machine until you decide to export them. That is sensible, because agent traces often hold sensitive data: prompts, tool arguments, screenshots, user files, access tokens.

There is one more curious piece — the failure store. AgentDebugX lets you save scrubbed bundles of the form "failure — diagnosis — fix" and then reuse them as debugging memory. That starts to look like an incident database for AI agents. One team fixes a rare case; another may later run into something similar.

The authors do admit, to their credit, that they have not yet measured the effect of that memory. The mechanism exists; its contribution to quality is still unproven.

What the experiments showed

The paper has two main questions.

First: can the system pinpoint the guilty step and the guilty agent more accurately?

Second: does that actually help repair failed runs, rather than merely produce good-looking reports?

For the first question they used the Who&When benchmark. Every trace in it is annotated with who made the mistake and when exactly the decisive failure occurred. It is a good test, because it demands more than noticing the problem — it demands localizing its causal source.

On the qwen3.5-9b model, DeepDebug posted the best result to date among the approaches compared. On the strictest metric — naming both the agent and the exact step correctly — it scored 28.8%. The best single-pass baseline managed 21.7%.

At first glance those numbers may look modest. But this is exactly the place not to expect magic. The task is genuinely hard: long traces, many intermediate actions, causality that is often smeared out. Against that background, a few percentage points is a noticeable gain in real diagnostic work.

Accuracy at identifying the guilty agent and the exact step, by trace length: DeepDebug's advantage is clearest on long trajectories.

The more interesting finding is elsewhere: DeepDebug's gain showed up most on long traces, those with more than 40 events. That makes sense. On short tasks one careful read is often enough. On long ones you need the second pass, the structural search and the comparison of hypotheses.

In tokens, the multi-round scheme turned out to be cheaper than you might assume. A single global pass averaged 8.1 thousand tokens; DeepDebug, 12.8 thousand. Roughly 1.6× more, not 5–10× more. The reason is that the later steps no longer read the whole trace, only narrow windows around the suspicious spots.

Does it actually fix tasks

The most applied part of the paper is the GAIA experiment. Here the authors took the Open-Deep-Research agent on qwen3.5-9b, which solved 55.8% of the validation tasks on the first attempt and failed 73 of them. They then tried to diagnose each failed run and replay it once.

The result is easy to read.

Baseline self-correction methods, which simply receive a general account of the failure, managed to repair between 4 and 6 tasks out of 73.

DeepDebug, with the cause localized and a specific edit attached, repaired 13 tasks.

Overall accuracy rose from 55.8% to 63.6%.

This is the question of whether diagnosis changes the final answer. In this case it does.

The gain is clearest on mid-difficulty tasks, the ones that require several reasoning and tool steps in a row. Precise localization of the error pays off there, because the agent is told exactly where the wrong fork was.

What DeepDebug is compared against matters too. The baseline correction approaches see the general context of the failure but get no precise localization of the causal step, no evidence and no formulated edit. Which is to say: knowing where the error is makes it markedly easier to fix.

Where the limits are

The work has limitations, and they are worth keeping in mind.

And that is probably the right framing. The authors do not promise automatic debugging as a magic button. They offer infrastructure in which diagnosis becomes systematic, checkable and repeatable.

Why this matters

If you build AI agents, the problem is no longer how to collect a long trace. The problem is how to get that trace to answer three questions:

AgentDebugX shows that these three questions can be tied into a single engineering loop. That matters more than it sounds. For now, debugging AI agents is mostly a cottage craft: somebody read the log, made a guess, tweaked the prompt, forgot about it. A process like that scales badly, verifies badly and leaves almost nothing behind.

The alternative proposed here: store failures as artifacts, write diagnoses for them, structure the re-run as a new branch, accumulate a memory of past errors, and gradually turn agent reliability into an ordinary engineering discipline.

The bottom line

AI agents fail tasks somewhere other than where the failure shows. Observability alone is therefore no longer enough. What is needed is a system that can walk from symptom to cause, from cause to fix, and then test the fix in a fresh run.

AgentDebugX builds exactly that loop. DeepDebug looks not for the step that failed but for the step that caused the failure. On the Who&When benchmark that yields more accurate attribution, especially on long traces. On GAIA that accuracy converts into extra tasks repaired after a single re-run.

In effect, the work moves AI-agent debugging toward a reproducible engineering procedure. And as agents acquire more tools, more memory, browser access and multi-agent coordination, that procedure stops being optional.