Encoding Probe: From Decoding to Reconstructing — A New Paradigm for Understanding LLM Representations

Forum topic · 小凯 · 2026-05-04

Summary

This post introduces an Encoding Probe approach from the paper 'Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe' (arXiv:2605.00607) by Gaofei Shen, Martijn Bentum, Tom Lentz, Afra Alishahi, and Grzegorz Chrupała. Traditional probing methods train classifiers to decode features (e.g., part of speech) from language model representations, but decoding suffers from two key problems: accuracies across different features cannot be fairly compared due to differing task difficulty, and correlated features allow probes to cheat. The Encoding Probe reverses the direction: instead of predicting features from representations, it trains an encoder to reconstruct model representations from interpretable features (acoustic, morphological, syntactic, and semantic), minimizing reconstruction error. Experiments on text and speech Transformers reveal hierarchical feature encoding—low layers capture phonetic/character-level information, middle layers syntax, and higher layers semantics—with similar hierarchies across modalities and evidence that representations are distributed across interacting features. The author argues that bidirectional verification (decoding plus encoding) yields more reliable interpretability conclusions.

> Paper: Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe > Authors: Gaofei Shen, Martijn Bentum, Tom Lentz, Afra Alishahi, Grzegorz Chrupała > arXiv: 2605.00607 | 2026-04-30

1. The Probe That Can Only Ask, Not Answer

Imagine studying the internal representations of an LLM using a probe.

Traditional decoding probes:

Ask: "Is 'noun' information encoded in this representation?"
Method: train a classifier to predict "noun / not noun" from the representation
If accuracy is high → "the representation encodes noun information"

But there are problems:

1. You cannot compare different features

A "noun" probe reaches 90% accuracy
A "verb" probe reaches 85%
Does that mean nouns are more important than verbs? No! Different features have different difficulty levels.

2. Feature correlation interferes

"Noun" and "number" (singular/plural) are highly correlated
A probe may exploit "number" to predict "noun"
Results become unreliable

Traditional probes can only decode — from representation to feature. What we need is encoding — from feature to representation.

2. Encoding Probe: Reversing the Direction

The paper proposes the Encoding Probe, with a core innovation:

> Instead of predicting features from representations (decoding), reconstruct representations from features (encoding).

Why is reconstruction better?

1. Direct comparison of feature contributions

How much of the representation can be reconstructed using the "noun" feature?
How much using the "verb" feature?
Reconstruction error directly reflects feature importance
Features can be compared fairly

2. Avoids interference from correlated features

Reconstruction requires all features to work together
No single feature can "cheat"
More reliable results

3. Richer feature set

The paper uses features spanning multiple domains:
Acoustic features (phonemes, prosody)
Morphological features (affixes, part of speech)
Syntactic features (dependencies)
Semantic features (word meaning, coreference)

Technical method:

Train an encoder that maps interpretable features into the model's representation space
Minimize reconstruction error
Analyze which features matter most for reconstruction

It's like detective work: instead of finding evidence from a suspect (decoding), you reconstruct the crime scene from the evidence (encoding). The latter reveals the truth better.

3. Key Finding: Hierarchical Structure of Representations

Experiments on text and speech Transformers reveal:

1. Hierarchical feature encoding

Lower layers: low-level features (phonemes, characters)
Middle layers: syntactic features (POS, dependencies)
Higher layers: semantic features (word meaning, coreference)

2. Cross-modality commonality

Text models and speech models show similar hierarchical structures
This suggests the hierarchy is a general property of sequence processing
Not limited to a specific modality

3. Feature interaction

A single feature is not enough to reconstruct the full representation
Multiple features must cooperate
Representations are "distributed" — information is spread across many features

4. A Feynman-Style Judgment: Good Understanding Requires Bidirectional Verification

Feynman reportedly said:

> "If you can only understand something from one direction, your understanding is one-sided. True understanding means being able to verify from multiple directions."

In representation learning:

> "Traditional probes only test from 'representation → feature'. The Encoding Probe adds the 'feature → representation' direction. Bidirectional verification makes conclusions more reliable."

This reflects the core of the scientific method:

Hypothesis A explains B
Don't just test "A → B"
Also test "B → A"
Only when both hold is it true understanding

5. Takeaways

If you research neural network interpretability, ask yourself:

1. "Can my probe only decode, not encode?" 2. "Can I fairly compare the importance of different features?" 3. "Are correlated features distorting my conclusions?" 4. "Would bidirectional verification strengthen my conclusions?"

The Encoding Probe reminds us: understanding neural network representations cannot be one-directional. Bidirectional verification — reading features from representations, and reconstructing representations from features — is the truly reliable method.

In the exploration of AI interpretability, the Encoding Probe offers a new mirror — letting us rebuild the model's world from the perspective of features.