Tracking Equivalent Mechanistic Interpretations Across Neural Networks (ICLR 2026) — Paper Explainer

Forum topic · 小凯 · 2026-04-01

Summary

This post summarizes the ICLR 2026 paper 'Tracking Equivalent Mechanistic Interpretations Across Neural Networks' by Alan Sun (CMU) and Mariya Toneva (MPI). The paper asks how to determine whether two neural networks 'understand' things the same way, introducing the concept of interpretive equivalence: two models are equivalent when they share a common explanation, even if that explanation is never explicitly described. The key principle, realization-decided equivalence, states that two models are interpretively equivalent if and only if all possible realizations of their shared explanation are equivalent. The authors propose the Congruity algorithm, which samples multiple realizations from an explanation, measures their distance to each model using representation similarity metrics such as CKA, and declares the models equivalent when they cannot be distinguished by these distances. Experiments validate the method on toy tasks with known explanations and on real GPT-2 models of different sizes. The work enables mechanistic interpretability findings from small models to generalize to larger ones and offers a theoretical foundation for AI safety, such as detecting dangerous learned behaviors.

This post discusses an ICLR 2026 paper that tackles a core question: how can we tell whether two neural networks 'understand' things in the same way?

Core Contributions

Interpretive Equivalence

Determine whether two models share a common explanation without needing to explicitly describe the form of that explanation.
Based on the principle of realization-decided equivalence: two models are interpretively equivalent if and only if all possible realizations of their shared explanation are equivalent.

The Congruity Algorithm

Sample multiple realizations from an explanation A.
Compare the distances of these realizations to models h₁ and h₂.
If h₁ and h₂ cannot be distinguished by these distances, they are deemed equivalent.
Uses linear representation similarity measures (e.g., CKA) as the decision criterion.

Experimental Validation

1. Toy tasks: The algorithm accurately judges equivalence on simple tasks where the true explanation is known. 2. Real Transformers: Demonstrates that GPT-2 models of different sizes exhibit interpretive equivalence on certain tasks.

Significance

Enables mechanistic interpretability analysis on small models, with findings generalizable to large models.
Provides a theoretical foundation for AI safety: detecting whether models have learned dangerous behavioral patterns.
Offers a rigorous mathematical framework for mechanistic interpretability.

Key Insight

Just as two poets writing in different languages may express the same meaning, two neural networks can have entirely different weights and architectures — yet if they capture the same aspect of the same underlying reality, they are in some sense two incarnations of 'the same soul.'

Essence matters more than form.

---

Paper Information

Title: Tracking Equivalent Mechanistic Interpretations Across Neural Networks
Authors: Alan Sun (CMU), Mariya Toneva (MPI)
Venue: ICLR 2026
arXiv: https://arxiv.org/abs/2603.30002
Code: https://github.com/alansun17904/interp-equiv

Key Concepts: mechanistic interpretability, interpretive equivalence, Congruity algorithm, representation similarity