Cammarata’s case against superposition divides interpretability researchers

The cat-and-car neuron may see a shape

Mechanistic interpretability tries to understand not just a model’s answers, but how it produces them. A long-standing puzzle is polysemanticity: neurons that appear to respond to several unrelated things. One famous example from the image model InceptionV1 responds to both cat faces and the fronts of cars.

The usual explanation is superposition. If a network has too few neurons for all the concepts it represents, the theory goes, it can pack unrelated concepts into the same neurons when they rarely interfere. Many labs have therefore trained a second model, such as a sparse autoencoder, to separate those concepts and study the result instead of the original network.

Cammarata helped shape this account. He co-wrote “Zoom In” with Chris Olah, an early description of the superposition framework, and launched OpenAI’s automated interpretability project in 2023. Even the most generous estimate then put the share of neurons researchers could understand at about 1.7%.

Now Cammarata argues that some canonical examples were misread. The cat-and-car neuron, he says, responds to V-shaped forms: pointed shapes such as fox faces or flames activate it, while rounded shapes like the letter O reduce its activity. On this account, the neuron does not represent two unrelated concepts and is not evidence of superposition.

He also reinterprets the “Zoom In” example in which a car seemed to be hidden among dog-responsive neurons. He calls it ordinary generalization: the diagram groups images of cars and dogs according to which way they face the camera. Confusion in InceptionV1’s later layers, he argues, reflects an outdated architecture and weak analysis tools.

Three synthetic visualizations of the same InceptionV1 neuron show cat faces, car fronts, and cat legs. "Zoom In" treated this mix as an example of polysemanticity. Cammarata now reads the neuron as a V-shape detector. | Image: Olah et al. / Distill, "Zoom In" (2020)

Source: the-decoder.com

Cammarata calls another pattern “recursive generalization.” A neuron, in his account, grows like a branch, developing side branches: one neuron first recognized the letter sequence “iy,” then responded to “diy,” and eventually to “do it yourself,” which contains no “iy.” Unlike superposition, he argues, these groups offer a way to understand how a network is organized.

The same two neurons, visualized two different ways. Better analysis methods make their response patterns and linked photos much easier to see. Cammarata says this is a reason to revisit older findings about how hard neural networks are to understand.

Source: the-decoder.com

His preliminary estimates for language models are striking. In Qwen3-32B, about 90% of neuron poles have one main function, around 7% have two related functions, and just 2–4% remain unexplained. Goodfire reports similar results in models with up to trillions of parameters. AI agents did most of the work, analyzing neurons at scale.

The approach is not a fundamentally new method. Cammarata returns to the field’s original idea—studying neurons directly—with better tools, larger models and automated agents. He invokes Rich Sutton’s “Bitter Lesson”: in interpretability, too, broad, simple investigation may outperform clever decomposition methods.

A diagram of "recursive generalization." A neuron for "iy" extends its responses through "diy" all the way to "do it yourself," linked by related meanings.

Source: the-decoder.com

What the results would mean for safety

If Cammarata is right, the shift could matter beyond interpretability research. Researchers have few ways to tell whether a model is deceiving people or behaving dangerously, beyond monitoring reasoning traces that are becoming less reliable. Cammarata doubts AI alignment—steering systems toward human goals—is possible without interpretability: otherwise, developers cannot know what they are building.

He argues that deception need not be hidden from a model’s internal structure. If people become another “reader” of the network, they may be able to spot deception too, and correct errors more directly.

An OpenAI employee who posts on X under the pseudonym Rune reacted with enthusiasm. Rune said he is close friends with Cammarata and might not have joined OpenAI without him. He suggested the work could be remembered as a major moment in science, and said even Eliezer Yudkowsky, one of the best-known writers warning of AI catastrophe, might have to lower his estimate of the chance of human extinction. Rune sees a future in which AI alignment becomes an ordinary scientific discipline.

But Rune also said the findings need independent confirmation. He warned that interpretability breakthroughs could speed up AI capabilities, and argued that results emerging outside major labs, including OpenAI, are not good news for those labs.

A useful tool is not a full solution

Neil Nanda, who leads Google DeepMind’s interpretability team and is considered one of the founders of mechanistic interpretability, is more cautious. After working at Anthropic and as an independent researcher, he became more pessimistic about ambitious plans for the field from mid-2024 and shifted toward practical methods. His team lowered the priority of sparse autoencoders—not because they considered them useless, but because the effort devoted to them seemed unjustified.

Nanda calls Cammarata’s work interesting, but not a change to the overall picture. Neuron analysis could help answer practical questions, such as why a model did something questionable, and is much easier to apply to frontier models. But Nanda does not think interpretability is close to a complete solution. He expects neuron analysis to have serious limitations and miss a great deal.

He still considers superposition real. Language models store many more facts than they have neurons, and much of that information may sit in weak activations. He doubts results from small image models will transfer to other models, and says Cammarata’s essay defines monosemanticity too loosely. Still, he credits Cammarata with revisiting earlier work using modern tools.

Some of the disagreement is about definitions. Cammarata does not object to models representing more concepts than they have dimensions; Nanda says they agree on that point. Cammarata’s objection is narrower: that networks arbitrarily pack completely unrelated concepts into the same neurons.

The accompanying technical work remains unpublished, and Goodfire has a direct interest in the hypothesis as an interpretability company. I think that makes independent replication the key test—not the headline estimates. If Cammarata’s account holds, automated researchers could do in months what took people nearly a decade on image models. The promise is not just a clearer picture of neural networks, but a race to get that picture before time runs out.