Goodfire essay argues neural networks are already largely interpretable
Nick Cammarata of Goodfire argues that the classic evidence for polysemantic neurons was misread and that most neurons in large models can be understood directly.
In the Daily Brief of 29 October 2026
Filed under AI safety, Large language models
In a long essay published by the interpretability company Goodfire and presented as a working hypothesis, Nick Cammarata argues that the canonical examples of polysemanticity, such as a neuron in the image model Inception V1 that responds to both cat faces and cars, were misinterpreted. In his account, what looked like arbitrary packing of concepts into neurons was mostly a pattern he calls recursive generalisation, produced by ordinary and largely fixable issues rather than by superposition.
The practical claim is that researchers can study a network’s own neurons and circuits directly, instead of first trying to extract a list of hypothesised features, and that most neurons in models up to trillions of parameters can now be understood. The essay draws on months of work on vision and language models.
Researchers responding to the essay have stressed that it offers limited formal evidence and that understanding complete circuits remains a distant goal.