Summary
A new arXiv paper (2605.15183) introduces tensor similarity, a weight-based metric for mechanistic interpretability that determines when two network components implement the same computation. Existing similarity measures either rely on empirical behavior—leaving them blind to out-of-distribution mechanisms—or on basis-dependent parameters, which ignores weight-space symmetries. Tensor similarity addresses both issues for tensor-based models: it is invariant to weight-space symmetries, captures global functional equivalence, and accounts for cross-layer mechanisms via an efficient recursive algorithm. Empirically, the metric tracks functional training dynamics such as grokking and backdoor insertion with higher fidelity than existing measures. Authored by ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe, Jacob Meyer Cohen, Logan Riggs Smith, and Thomas Dooms, the work provides a rigorous tool for verifying mechanistic equivalence in neural networks.
Paper Overview
Field: ML
Authors: ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe, Jacob Meyer Cohen, Logan Riggs Smith, Thomas Dooms
Published: 2026-05-14
arXiv: 2605.15183
Abstract (original)
Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measures evaluate either empirical behaviour, leaving them blind to out-of-distribution mechanisms, or basis-dependent parameters, meaning they disregard weight-space symmetries. To address these issues for the class of tensor-based models, we introduce a weight-based metric, tensor similarity, that is invariant to such symmetries. This metric captures global functional equivalence and accounts for cross-layer mechanisms using an efficient recursive algorithm. Empirically, tensor similarity tracks functional training dynamics, such as grokking and backdoor insertion, with higher fidelity than existing metrics. This redu...
---
*Auto-collected on 2026-05-15*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177620062