Cammarata argues that neural networks are already clean
In a long essay presented as a working hypothesis, Nick Cammarata of Goodfire argued that canonical examples of polysemantic neurons were misread and that superposition is not the main explanation.
In the Weekly Review of 1 November 2026
Filed under Large language models, AI safety
Polysemantic neurons, which respond to several unrelated features, have been a central obstacle for interpretability; the most famous one, in an image model, responded to both cat faces and the fronts of cars. The leading explanation has been superposition, in which a network stores more features than it has neurons. Nick Cammarata, of Goodfire, argued that many such examples, including the last layer of InceptionV1, were misinterpreted and have ordinary causes.
Cammarata presents the essay as a working hypothesis. If it holds, interpretability work could study neurons directly rather than first decomposing them into features.