Paper
Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe
- Authors: Gaofei Shen, Martijn Bentum, Tom Lentz, Afra Alishahi, Grzegorz Chrupała
- arXiv: 2605.00607 | 2026-04-30
- Ask: "Does this representation encode 'noun' information?"
- Method: train a classifier to predict "noun / not-noun" from the representation
- High accuracy → "the representation encodes noun information"
- A "noun" probe scores 90%, a "verb" probe 85%—does that mean nouns matter more? No: different features have different intrinsic difficulty. 2. Correlated features interfere
- "Noun" and "number" are highly correlated; a probe may cheat by using "number" to predict "noun," making results unreliable.
- Acoustic features (phonemes, prosody)
- Morphological features (affixes, part of speech)
- Syntactic features (dependencies)
- Semantic features (word meaning, coreference)
- Lower layers: low-level features (phonemes, characters)
- Middle layers: syntactic features (POS, dependencies)
- Higher layers: semantic features (meaning, coreference)
The Problem: Probes That Can Only Decode, Not Encode
When studying LLM internal representations, the traditional approach uses a decoding probe:
But this has serious limitations:
1. Cannot compare different features fairly
Traditional probes only go from representation → feature. What we need is the reverse direction—feature → representation.
Encoding Probe: Reversing the Direction
Core idea: Instead of predicting features from representations (decoding), reconstruct representations from features (encoding).
Why reconstruction is better:
1. Directly compares feature contributions — reconstruction error reflects how important a feature is, enabling fair comparison across features. 2. Avoids correlation interference — reconstruction requires all features to work together; a single correlated feature cannot cheat. 3. Supports rich feature sets — the paper uses features spanning multiple domains:
Technical approach: train an encoder that maps interpretable features into the model's representation space, minimizing reconstruction error, then analyze which features matter most.
*It's like detective work: rather than finding evidence from a suspect (decoding), reconstruct the crime scene from the evidence (encoding)—the latter reveals more truth.*
Key Findings: Hierarchical Structure of Representations
Experiments on text and speech Transformers reveal:
1. Hierarchical feature encoding
Bidirectional Verification
> "If you can only understand something from one direction, your understanding is one-sided. True understanding means verifying from multiple directions." — Feynman-style reasoning
In representation learning:
> Traditional probes only test representation → feature. Encoding Probe adds the feature → representation direction. Bidirectional verification makes conclusions more reliable.
This mirrors the scientific method: if hypothesis A explains B, test both A→B and B→A; only when both hold is understanding genuine.
Takeaways
If you work on neural network interpretability, ask yourself:
1. Does my probe only decode, never encode? 2. Can I fairly compare the importance of different features? 3. Do feature correlations contaminate my conclusions? 4. Would bidirectional verification strengthen my findings?
Encoding Probe reminds us: understanding neural representations cannot rely on a single direction. Reading features out of representations *and* reconstructing representations from features—together—form the reliable path forward.