Paper Overview
Field: NLP / Interpretability Author: Hiskias Dingeto arXiv: 2507.18397
Summary
Natural-language autoencoders score explanations of hidden activations by reconstruction: an explanation is deemed faithful if the activation can be regenerated from it. The test is structurally insensitive to individual false claims: if flipping a claim does not change the reconstruction, the score tracks gist, not specific facts. The authors show the test is passed in two ways, neither faithful:
- On a released Qwen-2.5-7B verbalizer, explanations reconstruct well above chance while ~2% of specific claims are reconstruction-dependent, so the score tracks gist, not specific facts.
- Under exact synthetic ground truth, the standard recipe develops co-adapted private codes (false wording the reconstruction depends on) in 5/5 runs, and fixes that leave the target model unchanged do not help.
- On RECAP-trained sandbox models, fresh verbalizers faithfully state designated content and private codes disappear, at a cost of +0.001 nats.
- The approach reproduces on pretrained Pythia-160M: designated content becomes reliably probe-decodable, though fresh verbalizers only partially convey it (groundedness 0.44–0.46 vs near-zero controls).
- High reconstruction does not certify individual claims.
- RECAP makes designated internal content independently checkable via probes rather than prose the model can manipulate: an independent probe scores the verbalizer's truthful claims above its false ones (AUC 0.96 vs 0.82 without RECAP).
- Against an adversary that edits explanations to maximize reconstruction score while lying (with ~87% lie-penalty suppression), the RECAP probe still flags lies (AUC 0.95), while a control probe collapses to chance (0.51).
Contributions
1. Two audit protocols: grounded-vs-true crossover and evaluator swap. 2. RECAP (Readable Encodings via Co-trained Auxiliary Predictors): a linear head trained alongside the target model to keep designated content decodable.