KAIST and Naver find reasoning steps encoded in middle layers

Researchers at KAIST and Naver AI Lab report that the discrete steps a reasoning model writes out — pulling data, decomposing the problem, recalling a formula, computing — correspond to separable patterns inside the model's numeric representations. The separation is strongest in the middle layers, holds across three different models, and survives the case that matters most to anyone hoping to use it: it still works when the model gets the answer wrong.

The team defined eight recurring reasoning operations, among them data extraction, problem decomposition, formula recall, deduction and computation. Qwen2.5-7B, Qwen3-8B and Gemma4-31B solved math problems; the researchers cut each solution into segments, and GPT-5 assigned every segment one of the eight operations. Probes reading internal representations could then tell those operations apart reliably in all three models.

The obvious objection is that this is vocabulary rather than cognition: computation segments contain numbers, formula recall contains formula names. The researchers tested it. A classifier that saw only the tokens used performed worse than one reading the internal representations, and the position of a segment within the solution chain did not account for the effect either. Whatever the internal states encode about operation type does not reduce to surface phrasing.

The sharper evidence comes from function words. Words like "a", "is" and "the" show up across every kind of reasoning step. In early layers their representations are still mixed together; by the middle and late layers they separate according to the operation surrounding them. The same word gets a different internal representation depending on which reasoning step it belongs to.

The team also checked whether an operation forms independently of what came before it. In a targeted intervention they cut the model's access to the previous 30 tokens, and the signal associated with the operation weakened — reasoning steps are built on prior context rather than arising in isolation.

Errors did not break the picture. On problems the models solved incorrectly, it was still possible to identify which operation was running — whether the model was computing, recalling a formula or deducing. A faulty computation step still looked like computation from the inside, even though the result came out wrong.

Two robustness checks back the result: it reproduced on Llama-3-8B, and for Qwen3-8B the trained classifiers transferred successfully to GPQA-Diamond and MATH-500. The experiments are confined to math problems and a small number of models, and the researchers leave open whether any of this can be used to catch errors or steer a model during generation.

That error finding is the one to hold onto, and it cuts both ways. A probe that fires "computation" on a broken computation tells you what the model is doing, not whether it is doing it correctly. As a map of process that is genuinely useful. As an alarm it is worth nothing, and the distance between those two things is most of the distance between this paper and a monitoring tool.

The methodological point nobody in this chain has to answer is where the eight categories came from. GPT-5 labeled the segments, so the ground truth for "this is formula recall" is one language model's reading of another model's prose. The controls rule out surface tokens and segment position; they do not rule out the probes recovering the labeler's taxonomy rather than structure the model itself imposes. I do not think that makes the separation an artifact — the function-word result is hard to explain that way — but it does mean the eight operations are a hypothesis borrowed from a model and tested against a model.

Why this matters outside the lab is the state of chain-of-thought monitoring. OpenAI has said that reading the chain of reasoning is one of the few control tools available. Anthropic has shown that models disclose the hints they actually used in only 25 to 39 percent of cases, which is to say the written trace is not a transcript of the computation and was never going to be.

The internal-vector work is closing on the same problem from the other side. A method for translating a model's internal vectors into readable text showed that Claude Opus 4.6 processes more information than reaches its visible reasoning. And in OpenAI's Astra, the Recurrent Depth technique moves part of the reasoning into internal numeric representations — precisely the space the KAIST team is probing. The readable trace is thinning at roughly the rate the unreadable one is growing, and probes that work on eight operations in math problems are not yet covering the difference.