AI agent frameworks leave gaps in real-time monitoring

Agents leave a longer trail

Unlike a standalone generative AI system that produces an answer for a person to review, an agent can plan, choose tools, launch subagents and act across a long sequence of tasks. That creates more ways for things to go wrong—and makes pre-deployment testing less able to anticipate every risk.

The authors and colleagues from academia, civil society and industry examined four widely used agent frameworks. They found gaps in signals needed to detect policy violations, unauthorized actions and failures of human oversight. Some frameworks lacked persistent agent identifiers and records of which people had approved an agent’s tasks. Without those records, it can be difficult to tell which agent did what or whether it had permission.

The authors argue that monitoring needs two kinds of records:

A mortgage application illustrates why both matter. An agent recommends rejecting an application, and a loan officer approves the decision. A bank or regulator may later need to establish which agent made the recommendation, who approved it and how long the process took. It may also need to examine the agent’s reasoning to check that it did not rely on prohibited information, such as the applicant’s neighborhood.

Logging can move faster than reasoning

Event records are largely a software problem. The authors point to OpenTelemetry, an open observability standard supported by engineers across companies and being adopted by agent-framework developers. Its working group is preparing shared formats for tracking agent activity, but the standards are not yet finalized.

The authors want those formats to cover persistent agent identification, human interventions and changes to permissions or memory. They also stress a privacy risk: an agent identifier could be linked to real people.

Decision records are harder. Researchers and model developers still need to establish whether a model’s displayed step-by-step reasoning reflects its actual decision process, and whether an agent’s confidence can be measured reliably. Those questions need answers before standards for using such records are set.

I think the distinction matters: logging actions is an engineering task with a plausible path through existing infrastructure; recording what an agent “thought” risks creating a false sense of visibility if the record is not a reliable account of how a decision was made.

Responsibility cannot stop at the framework

Framework developers—including OpenAI, Anthropic, CrewAI and LangChain—can build telemetry into their products. Deployers can press for missing records even before common standards exist, especially in regulated sectors such as banking and hospitals. Regulators and standards bodies should define what responsible deployment requires while leaving technical implementation to industry. In the US, NIST’s agent standards initiative has begun addressing agent identification, one of the gaps the authors found.

Organizations also need rules, permissions and roles that specify what agents may safely do. Telemetry alone cannot tell whether an action broke a particular rule.

The authors compare current agent deployment to sending a thousand bank employees to handle customer accounts without identification, then hoping to reconstruct events if something goes wrong. My guess is that observability will become a condition for trust, not a substitute for accountability: a record can show what happened, but rules and enforcement determine what follows. Until both exist, delegating work to an agent also means handing over some control of how that work is done.

Madhulika Srikumar is AI safety program lead at Partnership on AI, an international nonprofit partnership of industry, academic and civil-society organizations.

Claire Leibowicz is head of AI, trust and society at Partnership on AI and a doctoral student at the University of Oxford.