Today's AI agents keep their autonomy outside the model, not inside it

Where automation ends and a real agent begins

The word "agent" gets stuck onto almost anything in AI these days. A code-editor extension is an agent. A wrapper around an LLM with tool calls is an agent. A bot that clicks buttons in a browser is an agent too. But the authors of a long conceptual paper, Critique of Agent Model, put an uncomfortable and useful question on the table: what if almost nothing we currently call an agent actually is one?

This is not another paper with a fresh benchmark and a table where someone beats someone else by 2.3%. It is an attempt to put terms and architectures back in order. The authors work through what actually makes a system an agent, how "agentic" systems differ from systems with real autonomy, and propose a design of their own called GIC.

Why does that matter? Because everything downstream depends on the answer: how to build the next generation of AI systems, what to expect from them in the real world, and what not to fear prematurely. If today's "agents" mostly rest on external scripts, prompts and engineering scaffolding, then talk of "machine will" sounds very different than it would if that autonomy actually lived inside the model.

The core claim: not every "agent" is an agent

The authors draw a key distinction between two kinds of systems.

The first is what they call agentic systems. These are the practical solutions of today: an LLM wired up to tools, memory, a plan of steps, checks and execution loops. They can be genuinely useful and powerful. But their intelligence often sits not in the model itself but around it — in the surrounding code, the rules and a process written out in advance.

The second is systems with real autonomy. Here the things that matter have to live inside the system itself: long-horizon goals, a representation of itself, the choice of how to act, control over how deeply to reason, its own learning.

That, in the authors' view, is what separates good automation from a real agent.

The basic picture: an agent acts in an environment, takes in observations and tries to reach a goal.

On paper this can sound like philosophy. But the paper turns into engineering very quickly. The authors break "autonomy" into five concrete axes: goals, identity, decision-making, self-regulation and learning. And on each axis they show where current systems come up short.

Five places where today's AI agents fall short

The first problem is goals. Today's systems usually get a short prompt covering a single episode: do this, find that, click here. The moment the session ends, the goal is gone. For simple tasks that is fine. For genuinely long work — research, serious software development, physical work in the world — it is not enough. What is needed is not a command for right now but a long-lived goal that the system decomposes into subgoals itself and revises as it goes.

The second problem is identity. Right now an agent's role, constraints and capabilities are usually set by a system prompt, a config file or the set of tools it can reach. Put differently, the system knows who it is because a human wrote it down by hand. The authors argue that a more mature agent needs a model of itself: an understanding of its own capabilities, limits, roles and relationships with others in the environment. And that model should update with experience.

Two approaches to goals side by side: step-by-step external instructions versus internal decomposition of a long-horizon goal into subgoals.

The third problem is decision-making. The authors take issue with the popular idea that if you scale a single model far enough and give it enough reasoning, planning will simply emerge inside hidden states and long chains of text. Their claim: long reasoning is not planning. To plan in any real sense, a system needs a world model — a separate mechanism that predicts what happens after a given action.

The fourth problem is self-regulation. Even if a system has both a fast reactive mode and slower planning, the question remains: when to use which? Today that is settled crudely. Either the system always thinks at length, or it follows a rigid external script, or a human flips the mode by hand. The authors want that choice to become part of the agent itself.

The fifth problem is learning. Almost every current system is trained, then frozen, then shipped. If something goes wrong: fine-tuning, a new prompt, new scaffolding. The authors want more — an agent that can learn both from real experience and from internal simulations in a world model, and that decides for itself when to train and when to act.

Why the world model sits at the center

One of the strongest passages in the paper is a clean separation of two roles that get conflated constantly. There is a model that answers "what should I do?". And there is a model that answers "what happens next?".

The authors insist these are different things and cannot be fused without cost. A world model should learn to predict reality as accurately as it can. An agent should learn to choose actions in service of a goal. Throw both objectives into one pot and you get an elegant but dangerous confusion: the system stops predicting the world so much as bending its predictions toward the actions that suit it.

Hence their favorite term — simulative reasoning. The idea is simple: before acting, the agent can run several options through the world model in its head, weigh the consequences and pick the best one.

A reactive policy versus simulative reasoning: in the second case the decision rests on predicted consequences inside the world model.

This is an important fork in the road. Plenty of people today hope that long textual reasoning inside an LLM will substitute for planning on its own. The authors say it will not. Text can look convincing, but without grounding in the dynamics of the world it remains a story rather than a testable prediction.

What they propose instead: the GIC architecture

That critique is the foundation for the authors' own proposal — the GIC architecture. The name stands for goal, identity, configurator.

The scheme works like this.

First the agent takes in observations and turns them into an internal representation of the state of the world. Then the goal module takes the overall long-horizon goal and picks the current subgoal. In parallel, the identity module updates the agent's representation of itself. After that a dedicated configurator decides how to act right now: react fast, continue the existing plan, build a new one or, say, drop into a learning mode.

If planning is called for, a second loop kicks in: the agent simulates possible trajectories through the world model, scores them with a critic and picks the most useful one. If the situation is simple or urgent, a fast reactive action will do.

The GIC architecture: goal, identity and configurator govern when the agent plans and when it acts directly.

This maps closely onto the human intuition about two modes of thinking, fast and slow. But here the authors add a third layer — a meta level that decides when to run which mode.

The paper frames the approach as three systems:

What the paper gets right

The paper's main strength is how well it sets the frame. There is a great deal of marketing around AI agents right now and very little careful definition. Here the authors do the opposite: they try to separate the real architectural questions from the noise.

The second strong point is the split between the agent model and the world model. This is not terminological nitpicking. A practical idea sits behind it: if we want planning, debugging and safety, it helps to know which module failed — the one that predicted the world badly, or the one that chose a bad action.

The third is the take on safety. The authors make a bold move: they argue that a more modular, internally organized architecture may be not more dangerous but easier to control. Why? Because goals, subgoals, identity and reasoning modes become explicit and inspectable. You do not have to guess at what quietly emerged inside the black box.

But there are questions

For all the force of the idea, the paper reads in places more like a manifesto and a research program than a finished engineering system. There are plenty of formal statements and theorems, but very few full-scale experiments so far. The authors say so themselves: the details and the first prototypes are pushed into companion papers.

There is another question: how affordable any of this is computationally. A world model, planning, a critic, a configurator, an identity that updates, continuous learning — it sounds powerful, and expensive. Especially if the setting is not a toy environment but the real open world.

Finally, the criterion for "real agency" will stay contested. The authors draw a hard line between external engineering scaffolding and internal organization. That is a useful research stance, but the industry will be dominated by hybrids for a long time yet — part of the autonomy inside the model, part of it outside. And those hybrids may well turn out to be the most practical class of systems for the next few years.

Why this matters right now

On one side, the market is already selling "agents" as the next interface to AI. On the other, the public conversation keeps jumping to frightening scenarios about losing control. The authors offer a saner picture.

Their argument, restated: today's systems are usually far less autonomous than they look. But if we really do want to build more autonomous systems, the place to build them is not chaotic scaffolding code but an architecture where goals, self-regulation, learning and the world model are laid out explicitly and separately.

That shifts the question from "how do we add more tools to an LLM" to "how do we design a system that organizes its own behavior".

The takeaway

Critique of Agent Model is a paper about what we are actually building when we say the word "agent".

Its main contribution is not in numbers but in clarity. The authors show that today's AI agents are often elaborate pipelines rather than autonomous subjects of action. And they offer an alternative: the agent as a single system with internal goals, an evolving identity, simulative reasoning, self-regulation and continuous learning, working alongside a separate world model.

Will GIC become a standard? Too early to say. But as an intellectual framework the work is strong. It helps separate a hype label from a genuine architectural problem. And right now that is probably worth more than another lead on a benchmark.