Neural networks are famously black boxes. We can see every number inside them, but those numbers rarely mean anything to a human. In October 2023, Anthropic’s interpretability team published Towards Monosemanticity, showing a method to break a model’s internal activity into thousands of features that correspond to recognizable concepts. It was a significant step for mechanistic interpretability, the effort to understand how models compute rather than only what they output.
The problem: polysemantic neurons
A natural hope is that each neuron in a network represents one concept. In practice, many neurons are polysemantic: the same neuron activates for unrelated things — say, academic citations, English dialogue and Korean text. Studying individual neurons therefore gives a confusing picture.
Anthropic’s earlier work, Toy Models of Superposition (2022), proposed an explanation. A model needs to represent far more concepts than it has neurons. It can pack them in using superposition: representing each concept as a particular direction across many neurons, overlapping with others. This works because most concepts are rarely active at the same time. The cost is that individual neurons look like a jumble.
The method: dictionary learning with sparse autoencoders
If concepts are hidden directions, the task is to find them. The team trained a sparse autoencoder on the activations of a layer inside a small transformer language model (a one-layer model with a 512-neuron MLP layer).
A sparse autoencoder has two parts:
- An encoder maps the 512-dimensional activation into a much larger number of features — up to tens of thousands in their experiments.
- A decoder reconstructs the original activation from those features.
Training rewards accurate reconstruction while penalizing the number of active features, so that each input is explained by only a few features at a time. The hope is that each feature then corresponds to one interpretable concept — a monosemantic feature.
What they found
Many features were strikingly interpretable. Examples from the paper include features that fired on:
- DNA sequences;
- Arabic script;
- text written in Base64 encoding;
- legal language;
- specific patterns in code and URLs.
The researchers evaluated interpretability by having human raters and an automated method judge how consistently each feature matched a description, and found features to be far more interpretable than individual neurons.
They also found that features behave causally: artificially activating a feature changed the model’s output in the expected way, for example pushing it to produce the corresponding kind of text.
As they increased the number of features in the dictionary, broad features split into more specific ones, suggesting a hierarchy of concepts.
Limits
- A tiny model. The study used a one-layer transformer, far from frontier models.
- Imperfect reconstruction. The features did not explain all of the model’s behavior.
- Interpretation is labor-intensive. Thousands of features need labels, and some remain unclear.
What came next
In May 2024, Anthropic published Scaling Monosemanticity, applying the method to Claude 3 Sonnet, a production model. It found millions of features, including abstract and multilingual ones, and safety-relevant features related to deception, bias and dangerous content. In a demonstration, amplifying a feature associated with the Golden Gate Bridge made the model bring up the bridge in almost every answer. Other labs, including OpenAI and Google DeepMind, have published related sparse-autoencoder research.
Why it matters
Interpretability could eventually let developers inspect what a model represents, detect problematic internal patterns, and verify safety properties rather than relying only on testing outputs. Sparse autoencoders are not the final answer, but they turned an apparently hopeless tangle of neurons into something researchers can start to read.
Key takeaways
- Individual neurons are often polysemantic, responding to several unrelated concepts.
- Superposition lets a network store more concepts than it has neurons, as overlapping directions.
- Sparse autoencoders decompose activations into many features, each active only occasionally.
- In a small transformer, many features matched clear concepts such as DNA, Arabic script or Base64.
- Follow-up work scaled the method to a production model, finding millions of features, some safety-relevant.
Frequently asked questions
What does “monosemantic” mean?
Having a single meaning. A monosemantic feature activates for one coherent concept, unlike polysemantic neurons that respond to several unrelated things.
What is superposition?
A way for a neural network to store more concepts than it has neurons by representing each concept as a direction spread across many neurons, overlapping with other concepts that rarely occur at the same time.
Can this method make AI systems safer?
Potentially. If researchers can identify features linked to deception, dangerous knowledge or bias, they may detect or adjust them. It is still an active research area, and current methods explain only part of a model’s behavior.