More dynamism in an agent's workflow graph does not always pay off

How the various approaches optimize LLM agent workflows — from fixed templates to dynamic graphs that are assembled and rewritten on the fly.

The paper From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents takes on an important question about how LLM-based systems are built today. A system is no longer a single prompt but a whole executable workflow: the model searches for data, calls tools, runs code, writes to memory, checks itself with verifiers and redoes steps when it has to. Coding makes this especially easy to see: the assistant reads files, proposes edits, runs the tests and decides whether to keep going or stop.

Researchers at IBM propose looking at such systems as a computation graph, where the nodes are actions (an LLM call, retrieval, a tool call, validation, message passing) and the edges are dependencies and flows of information. That makes the architecture of the workflow something you can actually discuss: how many steps there are, who talks to whom, where the checks sit, what it costs in tokens, time and money.

What the problem is

The workflow is often treated as secondary: there is some skeleton, and after that you simply fill it with prompts and tools. In practice it is the structure that usually decides whether the system will be reliable and cheap. Add a verifier and quality goes up, but so does the cost. Go multi-agent and sometimes it gets better — and sometimes the communication inflates the budget and control over execution is lost.

Why systematize all this

The goal of the survey is a common framework for comparing workflow optimization approaches even when they look nothing alike: from compiling prompts inside a fixed pipeline to generating a new graph for every request. For that it introduces the notion of an agentic computation graph (ACG) and insists on a distinction between three levels:

First, the graph template — the reusable design that exists before the run.

Second, the realized graph — what actually got assembled and executed in a particular run (including branching, disabled nodes and checks that were added).

And third, the execution trace — the factual record of steps, observations and costs.

The triad is useful because it lets you separate honestly what we designed from what really happened at inference time.

From static templates to graphs

Static methods optimize the template before deployment: you can compare, hold a budget and reproduce results more easily. Such work usually improves either the topology itself or the parameters of the nodes — prompts, model choice, tool and verifier settings. The strength of the static approach is predictability; the weakness shows up when data and tools drift, and when the environment behaves unexpectedly.

Dynamic methods, by contrast, select, generate or edit the graph for a specific run — either before execution or right in the middle of it.

The researchers argue that more dynamism does not always mean a better result. Sometimes a simple choice of subgraph is enough to cut cost sharply. And sometimes there is no way around editing the graph as it runs, because it is the intermediate results that reveal what to do next.

What it adds up to

The authors collected 77 relevant papers and separately laid out 27 resources that can be used to evaluate workflow optimization approaches. And what has to be evaluated is not only the final quality but the properties of the graph and the execution — cost, depth, the volume of communication, the number of edits, how much the structure varies across different inputs, and robustness to tool failures and API drift.

So when a paper claims "our approach is better" but does not show which graph actually ran and at what price, the comparison often means nothing. The improvement may be no more than the result of extra compute, hidden retries or more expensive checks.

What this means for people building LLM agents

Start with simple scaffolding and tune the nodes (prompts, tools, validation) until you have a baseline. If the tasks differ a lot, the sensible first move is dynamic subgraph selection — it often buys a large saving in cost with almost no loss of reliability. Only after that go on to generating or editing the graph on the fly, because that is more expensive, harder to validate and demands explicit budget limits and stopping rules.