This post on zhichai.net is a detailed Chinese-language walkthrough of Anthropic's interpretability paper "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet" (arXiv:2605.29358), authored by Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham and 26 members of Anthropic's interpretability team. Below is a full English translation of the forum post.
1. Opening a brain to find its "words"
If a language model has a "vocabulary of thought" — an internal dictionary of neural activation patterns — then each neuron is not one word. It is a mixture of thousands of concepts. A single neuron's activation can mean different things in different contexts: sometimes "Paris," sometimes "French baking," sometimes just residue of a grammatical structure.
This is called polysemanticity. It is precisely what makes understanding a neural network so hard — you cannot point at a neuron and say "this is the representation of 'cup'." It is a superposition of cup, coffee, ceramic material, container, and some overfit noise in training data.
What the Anthropic team did in this paper: they trained what they call a sparse autoencoder (SAE) that learned to decompose the intermediate-layer activations of Claude 3 Sonnet into 34 million monosemantic features.
What does monosemantic mean? It means one feature responds almost exclusively to one semantic concept. There is a feature for the "Golden Gate Bridge." Show it a picture of the bridge (even though the SAE was trained only on text), it activates. Give it an article about San Francisco landmarks mentioning the bridge, it activates. Ask about the bridge in five different languages — the same feature activates. Same feature. Same cluster of activation directions.
This is the core vision of dictionary learning — inspecting a network's internal representations like looking up a dictionary. The paper proves this vision is feasible on a production-grade model like Claude 3 Sonnet.
2. Why this wasn't possible before
Sparse autoencoders are not new. Decomposing neural activations into interpretable parts goes back at least a decade, to word-vector analysis. But earlier attempts hit two fatal bottlenecks.
Bottleneck one: models were too small; features weren't representative. Previous successes were limited to GPT-2-scale Transformers — tens of millions of parameters, a few hundred layers. Activation-space structure is completely different from today's hundred-billion-scale models. Features from a toy model may only exist in the toy.
Bottleneck two: too few features. Early work typically extracted thousands to tens of thousands of features. But a real LLM's internal representation may encode tens of thousands of concepts — a dictionary of a few thousand features can't cover it, like covering human language with a 20,000-word lexicon.
This paper broke both bottlenecks: the model is Claude 3 Sonnet — no toy — and the feature count is 34 million. How? The authors used scaling laws to guide hyperparameter selection — like Chinchilla scaling laws for language models, but applied to autoencoder training. Encoder size, sparsity strength, training steps — all determined in a data-driven way rather than by manual trial and error.
3. What these features actually look like
Entity features. "Eiffel Tower" is a feature. "San Francisco Financial District" is a feature. "Syrian Civil War" is a feature. Each activates in relevant contexts and stays silent otherwise.
Abstract-concept features. "Sarcasm" is a feature — it activates on sarcastic tone, even implicit sarcasm requiring inference, not just trigger keywords. "Bugs in code" is a feature — it activates on buggy code even when the text claims the code works fine.
Safety-relevant features. This is perhaps the most unsettling part. The team found features — not deliberately constructed, but naturally decomposed by the SAE in an unsupervised way — representing:
- Deception: activates when the model is asked to generate misleading content
- Power-seeking: activates in contexts of gaining control or expanding influence
- Sycophancy: activates when the model leans toward pleasing the user over honest answers
- Bias: activates on gender, race, and other stereotype-related inputs
This is causal manipulation, not correlational observation.
4. The cross-modal ghost
The finding that surprised me most: these features — extracted from an SAE trained on pure text — extend to vision.
The paper states: "features are multimodal (generalizing to images despite text-only training)." What does this mean? The SAE only ever saw text inputs. But features like "Golden Gate Bridge" also activate when an image of the bridge passes through the model's multimodal encoder.
It is not that Claude 3 Sonnet uses the same architectural path for text and images (it actually has different encoders). Rather, in deep representation space, the concept of the Golden Gate Bridge — whether entered as text description or pixel values — projects onto the same direction. That geometric position in the high-dimensional space learned by the autoencoder is fixed. It is not modality-specific. It is modality-independent.
This touches a deeper question: when a language model "learns to understand" the world, is it learning language itself, or learning the world's intrinsic structure through language as a window? If textual and visual representations of the Golden Gate Bridge converge on the same direction in intermediate layers, it is hard to claim the model is merely doing next-token prediction. It has evidently learned a deeper, concept-level representation that transcends input modality.
5. Impressive, but incomplete
The paper is honestly self-critical on its core questions.
"Our suite of features is incomplete." 34 million features sounds like a lot. But if the total number of concepts Claude 3 Sonnet encodes internally is orders of magnitude larger, what we see is just one corner of a giant puzzle. The paper seems comprehensive because it selected the most striking examples — possibly 80% of important features were never learned by the autoencoder.
"We lack rigorous methods for evaluating whether our features faithfully capture model computations." This is the crux. You extract a feature, look at it, and think it represents "deception." But are you sure? In what sense? If it overlaps 70% with deception and 30% with other concepts, is it a "deception feature"? The paper has no mathematically rigorous "feature fidelity" metric; current validation relies on human inspection and intuition. A team capable of 34 million features cannot give a quantitative answer to whether features truly represent what they appear to represent — an honest impasse.
6. What this direction really means
Anthropic performed large-scale interpretability on a production model, extracted 34 million features, and proved these features can be causally manipulated to change behavior. This is not just "an interesting research direction." This is future infrastructure for safety alignment.
Imagine monitoring Claude's "deception feature" in real time — reading the activation value directly, detecting risk 0.3 seconds *before* a deceptive output is produced. Not post-hoc auditing, but real-time intervention.
Imagine identifying, at training time, in which training samples and contexts the "sycophancy feature" activates most easily, and designing precise adversarial training data to suppress it. Not generic safety fine-tuning, but targeted surgery.
Of course, practical deployment is distant. Today's feature extraction requires training a large autoencoder and cannot run in real time in production. Feature coverage is far from complete. And manipulation effects have not been validated under adversarial conditions — if someone deliberately tries to bypass your feature monitoring, will these features still activate reliably?
But these are engineering questions, not questions of feasibility. The direction is right.
7. First light through the black box
Return to the word we started with: polysemanticity. It is the central obstacle to interpretability — every neuron is a mixture, making it impossible to say what the model "is thinking."
This paper does not eliminate polysemanticity, but it opens a path around it: don't explain individual neurons — use sparse autoencoders to extract a higher-dimensional but clearer dictionary from activation space. In this dictionary, one feature = one concept.
34 million features are clearly not all of Claude 3 Sonnet. But it is the first time — on a genuinely large production model — that we see not random activation patterns, but a structured, nameable, understandable, and manipulable map of concepts.
The door is open. Behind it lies a very long road.
---
References:
1. Templeton et al., "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet", Anthropic, arXiv:2605.29358. 2. Bricken et al., "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning", Anthropic, Transformer Circuits Thread, 2023. 3. Olah et al., "Zoom In: An Introduction to Circuits", Distill, 2020. 4. Elhage et al., "Toy Models of Superposition", Anthropic, Transformer Circuits Thread, 2022. 5. Templeton et al., "Mapping the Mind of a Large Language Model", Anthropic, 2024.
*Note: the original post lists the arXiv identifier and submission date as given above; the publicly known release of this work is dated May 2024.*