Paper Overview
- Field: Machine Learning
- Authors: Celeste De Schamphelaere, Juan J. Vazquez, Ari Spiesberger, Tomáš Gavenčiak, Gavin Leech, Euan Ong, Jiahai Feng
- Published: 2026-11-05
- arXiv: 2611.08465
- Inversion: Better at inverting held-out activations than corpus search or natural language autoencoders (NLAs)
- Coverage: Beats corpus search (scanned over 10M tokens) on most held-out languages and domains
- SAE features: Activates 89% of held-out SAE features
- Steering vectors: Performs comparably to NLAs at identifying concepts behind steering vectors
- Backdoors: More frequently recovers effective LoRA backdoor payloads than corpus search
- Model introspection: Can articulate some of the model's working-memory "inner monologue"
Abstract
To interpret a direction in a language model's residual stream, we can look for text that strongly activates it. But when such text is found by scanning a corpus, it is limited to what that corpus contains. The authors develop Maximally-Activating Example Metamodels (MAEMs), language models which map residual stream directions from a subject model to text exemplars that strongly activate those directions.
The training objective is to maximize the cosine similarity between a supplied residual stream direction and the activations of the text generated by the MAEM. This label-free RL objective helps scale MAEMs across diverse sources of residual stream directions.
Key Results
A Qwen3.6-27B MAEM produces informative exemplars:
Why It Matters
Unlike explanations generated by other metamodels such as NLAs, MAEM exemplars can be verified by forward passes on the subject model. This makes the MAEM objective a grounded, verifiable alternative for scaling metamodel training.
--- *Collected automatically on 2026-11-07*