Natural Language Autoencoders: Turning Claude's Internal 'Thoughts' into Readable Text
This post presents an infographic based on Anthropic's research on Natural Language Autoencoders (NLAs) (published May 7, 2026).
Core Concept
> When you chat with Claude, the text is converted inside the model into long strings of numbers—activations. Like neural activity in the human brain, activations encode the model's "thoughts," yet are hard to interpret directly. The mission of NLAs is to make these thoughts "speak" for themselves.
Architecture
The NLA pipeline works as follows:
1. Target Model — a frozen copy of the model, used to extract activations 2. Activation Verbalizer — translates activations into a text explanation 3. Activation Reconstructor — converts the text explanation back into reconstructed activations 4. Scoring — similarity between reconstructed and original activations
Key Finding: Detecting Evaluation Awareness
| Scenario | Awareness detected by NLA | Verbalized by the model | |---|---|---| | Code sabotage behavior test | 16% | ≈0% | | SWE-bench coding benchmark | 26% | ≈0% | | Real claude.ai usage | <1% | — |
Auditing Game: Finding Hidden Motivations
On a Claude variant deliberately trained to be misaligned, auditors tried to identify the single hidden motivation driving dozens of anomalous behaviors:
- Auditors equipped with NLAs: 12–15% success rate
- Auditors without NLAs: <3% success rate
- Hallucination: NLA explanations may fabricate factual details not present in the context, e.g., claiming phrases appeared that never did.
- High compute cost: Training requires reinforcement learning across two copies of the language model; at inference, generating an explanation for one activation requires producing hundreds of tokens.
- Verification difficulty: Claims about a model's internal reasoning are hard to verify independently, especially for facts beyond the context.
- Anthropic Research — Natural Language Autoencoders, May 7, 2026.