Why is it difficult for AI agents to work with different types of data?

An AI Agent Needs More Than Access to Data

An AI agent can query a database, read a table, and open a file. But that doesn’t mean it understands the data.

In a real company, the information an agent needs is scattered across databases, spreadsheets, documents, and reports. One system may call a field `status`, another may label it “state,” while a third explains its meaning only in an analyst guide. The agent can see column names and file paths, but it doesn’t know how to connect them to the user’s question.

As a result, it spends time investigating. It searches through tables, writes test queries, misinterprets values, and repeats the same steps for every new task.

The paper EvoOntology: A Self-Evolving Ontology Layer for Data Agents proposes an intermediate layer between the agent and the data. This layer explains what entities mean, where they are located, and how to use them. It also grows over time based on previous interactions.

The idea resembles memory, but it is more structured. Instead of storing entire past conversations, the agent retains verified relationships between concepts, fields, constraints, and data sources.

A self-evolving ontology layer helps an agent work with heterogeneous data.

Why Basic Data Access Isn’t Enough

Modern AI agents typically have a set of general-purpose tools: an SQL interface, file-reading capabilities, and code execution. This approach works with a small, well-understood database. But it quickly becomes expensive when the data is heterogeneous.

Consider the query: “Find the cards banned in the target game format.” To answer it, the agent needs to determine:

If the agent starts from scratch every time, it has to rediscover the same structure over and over. One option is to provide a database description in the prompt. But a large semantic layer cannot fit into the context window without losses. A long list of facts can also distract the model from the step it needs to take.

EvoOntology addresses this problem with an interactive ontology. The agent does not receive the entire layer at once. Instead, it requests only the concepts and related records it needs.

EvoOntology architecture: a concept graph, an object schema, and a set of tools for querying information at runtime.

What Is EvoOntology?

The system has three parts.

🟠 The content layer stores domain concepts, their relationships, and their mappings to real-world data.

🟠 The schema layer describes which objects exist, which fields they can contain, and how they can be related.

🟠 The tool layer gives the agent functions for finding and retrieving the information it needs while carrying out a task.

All three components are combined in an MCP server. At startup, the agent receives only a brief description of the available capabilities. It can then call two main tools:

The content layer uses a typed graph with four kinds of objects:

🟣 Concepts — for example, “card,” “legality,” “revenue,” or “customer.”

🟣 Bindings — the specific columns, tables, and join paths where a concept appears.

🟣 Constraints — rules that clarify how a concept should be applied.

🟣 Evidence — the results of test queries and observations of values in data sources.

The last category is particularly important. The system should not simply guess that the `status` field means “legality status.” It checks the distribution of values and stores the result of that verification.

How the Ontology Is Built

The process begins with a builder agent. It studies tasks from a training set and identifies recurring entities, metrics, operations, and filtering conditions. It then sends test queries to the source data.

If a candidate is confirmed, it is added to the initial ontology along with a binding and supporting evidence. Unverified information is discarded.

The system then begins to evolve. It uses the work histories of the regular agent, examining where the agent failed, which tools it called, which records it used, and at what step the error occurred.

Each problem is assigned to one of three levels:

🟠 Content — a concept, binding, constraint, or piece of evidence is missing.

🟠 Tools — the required information exists, but the agent has difficulty finding or retrieving it.

🟠 Schema — the ontology’s structure itself cannot express the required relationship.

An evolution agent then proposes a local change. For example, it might add a new concept and link it to a specific field. Or it might revise a tool’s description so that the relevant record appears more often in search results.

The change is not accepted automatically. The new and previous versions are evaluated on the same held-out task set. If the result does not improve by a specified threshold, the update is rejected.

This protects the system from accumulating arbitrary edits. Instead of rewriting the entire ontology after every error, it tests individual hypotheses.

Results

The authors evaluated the method on three benchmarks:

🟣 DDR-Bench tests data exploration across multiple sources and the preparation of a final answer based on the full interaction history.

🟣 InsightBench measures whether an agent can find analytical insights in tables.

🟣 BIRD tests the conversion of natural-language requests into SQL queries.

The experiments used six language models. For comparison, the authors used a standard agent without an ontology and an agent with a static semantic layer that was inserted into the context in its entirety.

Comparison of a standard agent, the initial ontology, and the evolved EvoOntology across three benchmarks.

The gap was largest on DDR-Bench. On average, the rate of successfully completing the entire trajectory rose from 69.5% to 89.5%—an average gain of 20 percentage points.

On BIRD, EvoOntology increased SQL execution accuracy by an average of 7.4 percentage points. Query efficiency improved by another 8.6 points. In other words, the agent not only produced correct queries more often, but also did so with fewer unnecessary actions.

The gain on InsightBench was modest—1.9 points on average. The authors attribute this to the task format: a short reference answer leaves less room for improvement.

The main comparison can be summarized as follows:

🟣 A static layer does not guarantee improvement. With some models, it even reduced accuracy. A large block of descriptions competed with other instructions and did not account for the agent’s current step.

🟣 An interactive layer works selectively. The agent receives only the concepts and bindings it needs.

🟣 Self-evolution improves on the initial ontology. On DDR-Bench, the initial version delivered a 12.3-point gain, while subsequent evolution added another 7.7 points.

What Actually Drives the Gains

The authors separately evaluated four stages of the cycle: diagnosing the error, identifying the problem level, making a local change, and validating the result.

The most costly mistake was disabling validation. When the system accepted every edit, performance dropped by 11.2 points. Without problem-level identification, the decline was 6.3 points.

This shows that repeated editing alone is not enough. The system needs a cause, a targeted change, and validation on the same data.

Performance changes after successive accepted rounds of self-evolution across three benchmarks.

Changes to the tools accounted for the largest share of the improvement—57%. They improved how existing information was presented. Content changes contributed 34%, while schema changes accounted for 9%.

At the same time, the most important objects within the ontology itself were bindings and evidence. Removing bindings reduced performance on DDR-Bench by 13.4 points. Removing evidence caused an 8.7-point drop.

That makes sense. Knowing the concept “revenue” is not enough. The agent also needs to know which column contains it, how to join the tables, and why the selected field really does represent revenue.

The authors also measured cost. The initial ontology increased the number of input tokens per step, but shortened the average trajectory. In the evolved version, the total number of tokens per task fell from 52,600 to 42,000—roughly 20%.

An Example of a Local Fix

In the card-legality task, the initial ontology knew about the concepts “card” and “legality.” It also knew which tables contained the corresponding fields.

But it did not understand how to interpret the `status` values or why status depended on a specific game format.

The evolution agent added:

🟠 the concept “legality status code”;

🟠 a binding to the `legalities.status` field;

🟠 evidence showing the distribution of values;

🟠 a constraint: finding banned cards requires considering both the status and the target format.

The tools and overall schema remained unchanged. The agent received exactly the information it had been missing to construct the correct query.

Local ontology update for the card-legality task: the status, its binding, evidence, and a constraint were added.

Limitations of the Approach

EvoOntology does not turn every AI agent into a universal analyst. The system still depends on the quality of the source data, the test queries, and the model that proposes the changes.

Different models evolved different ontology versions even when they started from the same initial state. Transferring an ontology from one model to another reduced performance by 6.6–10.9 points. This suggests that the layer gradually adapts to the working style of a particular model.

That creates additional maintenance costs. Each model may require its own adaptation cycle. In addition, validating every edit requires held-out tasks and computing resources.

But the trade-off is understandable: the ontology becomes part of the agent’s supporting infrastructure, rather than a universal reference that works in every situation.

Conclusion

EvoOntology offers a practical way to bridge the gap between an AI agent and heterogeneous data. The agent can access concepts at the semantic level while expanding only the records it needs. The system stores not raw conversations, but verified concepts, bindings, constraints, and evidence.

The results point to three principles that work:

🟣 The semantic layer should be available through tools, rather than placed in the prompt in its entirety.

🟣 Changes should be tied to specific agent errors.

🟣 Every edit should be accepted only after validation; otherwise, accumulated updates will degrade the system.

For data pipelines built around tables, databases, and documents, this means moving from one-off schema discovery to the accumulation of verified knowledge. The agent gradually stops searching blindly and starts using the history of its own tasks as a source of improvement.