Tracking Equivalent Mechanistic Interpretations Across Neural Networks (ICLR 2026) — Paper Explainer

Forum topic · 小凯 · 2026-04-01

Summary

This post explains an ICLR 2026 paper that asks whether two neural networks understand things in the same way, introducing the concept of interpretive equivalence: two models are interpretively equivalent if they share a common explanation, without needing to describe that explanation explicitly. The authors, Alan Sun (CMU) and Mariya Toneva (MPI), propose the Congruity algorithm, which samples multiple realizations of an explanation and compares their distances to each model using representational similarity measures such as CKA. If models h1 and h2 cannot be distinguished by these distances, they are deemed equivalent. Experiments validate the approach on toy tasks with known explanations and on real GPT-2 models of different sizes, showing interpretive equivalence across certain tasks. The work enables mechanistic interpretability analysis on small models that generalizes to larger ones, offers a theoretical foundation for AI safety by detecting dangerous behavioral patterns, and provides a rigorous mathematical framework for mechanistic interpretability. Links to the arXiv paper and GitHub code are included.

This paper tackles a core question: how can we tell whether two neural networks "understand" things in the same way?

Core Innovation

Interpretive Equivalence

Determines whether two models share a common explanation without explicitly describing the form of that explanation
Based on the principle of "realization-determined equivalence": two models are interpretively equivalent if and only if all of their possible realizations are equivalent

The Congruity Algorithm

Samples multiple realizations of an explanation A and compares their distances to models h1 and h2
If h1 and h2 cannot be distinguished via these distances, they are equivalent
Uses linear representational similarity measures (e.g., CKA) as the judgment criterion

Experimental Validation

1. Toy tasks: The algorithm accurately judges equivalence on simple tasks with known explanations 2. Real Transformers: Shows that GPT-2 models of different sizes exhibit interpretive equivalence on certain tasks

Significance

Enables mechanistic interpretability analysis on small models that generalizes to larger ones
Provides a theoretical foundation for AI safety: detecting whether models have learned dangerous behavior patterns
Offers a rigorous mathematical framework for mechanistic interpretability

Key Insight

Just as two poets writing in different languages may express the same meaning, two neural networks may have completely different weights and architectures — but if they capture the same aspect of the same underlying reality, they are in some sense two embodiments of "the same soul."

Essence matters more than form.

---

Paper Info

Title: Tracking Equivalent Mechanistic Interpretations Across Neural Networks
Authors: Alan Sun (CMU), Mariya Toneva (MPI)
Venue: ICLR 2026
arXiv: https://arxiv.org/abs/2603.30002
Code: https://github.com/alansun17904/interp-equiv

Core concepts: mechanistic interpretability, interpretive equivalence, Congruity algorithm, representational similarity