What LLMs lack for AGI is a coordination layer, not understanding

There is a lot of argument around AGI right now: LLMs supposedly just predict the next word with some probability, so you cannot build "general AI" on them. The authors of The Missing Layer of AGI: From Pattern Alchemy to Coordination Physics from Stanford suggest looking at the problem differently. Their argument is that LLMs really do not hand you general intelligence out of the box — but not because they lack real understanding or thought. What they usually lack is a separate module that organizes reasoning as a process: sets the goal, picks the tools, checks the steps, holds the plan and the memory.

Instead of the familiar dilemma — either scaling solves everything, or LLMs are an evolutionary dead end — the paper offers a third position: the LLM is a powerful foundation (in the cognitive metaphor, something like fast System 1), and on top of it you need a System 2 coordination layer. And that layer can be described, measured and engineered.

An ocean of patterns

The authors offer a fishing metaphor. Inside an LLM there is an "ocean" — a huge store of patterns, associations and statistical habits accumulated from data. Ask a question without enough to hold it in place, and the model does exactly what "next-token prediction" training set it up to do: it returns the answer that looks most plausible given past experience.

Real reasoning appears when the system can deliberately "lure" the structures it needs to the surface while filtering out the rest. That means not just generating, but steering generation: pinning meaning down with external constraints, testing whether it holds, and not losing the thread on long tasks.

How the researchers propose to formalize it

The paper's central idea is semantic anchoring — any external support that ties the model's output to the task and its constraints: examples, retrieval, tool outputs, goal clarifications, checkable facts. The authors argue that the effect of anchors is often nonlinear: a little context may change nothing, but past a certain threshold the system's behavior switches sharply into a more goal-directed mode.

To describe this they introduce an anchoring strength S, which depends on three things: how well the anchors actually support the target concept (ρ_d), how unstable the representation is and how easily it drifts when the prompt is reworded (d_r), and how many anchors we drag into the context (k), with a penalty for excess context (γ log k). The important part is that anchoring costs resources, and putting more text in the prompt is under no obligation to help.

From theory to architecture

To keep this from staying philosophy, the authors turn the idea into a sketch of an architecture called MACI (Multi-Agent Collaborative Intelligence). The point is that a coordination module is easier to build as a system of roles than as one monolithic LLM: one part generates hypotheses, another argues and widens the search, another judges by Socratic rules and throws out badly posed arguments, while memory records commitments and intermediate results so the system can roll back and repair local errors instead of rewriting everything from scratch.

This is where the research sounds most practical: many of the typical LLM failures come down to the absence of engineering discipline around reasoning. What is missing is checks, ways of handling disagreement, durable memory, tool-based verifiers, stopping rules. The authors suggest treating these as diagnosable coordination failures: can we raise ρ_d, lower d_r, spend the k budget sensibly, and push the system across the threshold?

Why a "cat" anchors more easily than a "pangolin"

An interesting detail: anchoring is not equally hard for every concept. If an object sits close to common prototypes, the system stabilizes the representation more easily (roughly speaking, a lower d_r). If the object is rare, unusual, poorly connected to familiar templates, you need either stronger anchors or "bridges" — intermediate descriptions and analogies that link the new thing to what has already been learned.

What this changes in the AGI conversation

The paper's main conclusion is a calm one and, arguably, useful for the industry: LLMs are not a dead end for AGI. They are a strong base, but without a coordination layer we will keep running into a system that is remarkably smart one moment and suddenly unable to solve simple tasks the next. The authors propose treating those switches as a tractable research area: build measurable anchoring mechanisms, train policies for managing debate inside a multi-agent system, design memory for reasoning, add grounding through multimodality and tools, and use symbolic components as verification loops rather than as a replacement for the LLM.

This is not a finished recipe for AGI, but it is a framework you can work with: what we are missing, and which levers can be turned so that next-token prediction becomes reliable, checkable work on long tasks.

My own view is this: an LLM is only the text modality. The AGI problem lies not in text itself but in a system's ability to pick the next token as a universal action for solving tasks — including through multi-agent interaction. What becomes decisive is the ability to act multimodally: to work with audio, video, bodily signals, 3D environments. All of these forms of perception and action are latent spaces just as text is, only with a different geometry and dynamics. AGI begins where a single space of meaning appears, one that allows movement between modalities and the choice of an action. That is why the LLM is a necessary base on the way to real world models.

And in that sense Ludwig Wittgenstein put it well:

"The limits of my language mean the limits of my world."