This is an English translation/summary of a Chinese forum post interpreting the paper *Multimodal Model Diffing for Feature Discovery and Control*.
Paper info
- Title: Multimodal Model Diffing for Feature Discovery and Control
- Authors: Hunar Batra, Lachin Naghashyar, Ashkan Khakzar, et al.
- Affiliations: University of Oxford, Microsoft
- arXiv: 2608.09928
- Causal removal: ablate a feature and observe whether a specific capability degrades
- Steering: amplify or suppress a feature to raise/lower the corresponding capability
- Geometric redirection: low cosine similarity of decoder directions = large change
- Visual energy: average activation strength under visual input
The intuition
Imagine a bilingual person's brain: when moving from a monolingual to a bilingual environment, which neurons get 'rewired'? Which formerly language-only neurons start processing visual information? If we could precisely localize these changes, could we control capabilities—or even block unsafe behaviors? MMDiff is a scalpel for this: it tracks how multimodal training changes a language model's internal structure.
Method, step by step
Stage 1: Sparse autoencoders (SAEs) — a microscope for AI
Neural networks are black boxes. SAEs decompose activations into interpretable feature directions, e.g. 'red round object', the word 'danger', or the spatial relation 'above'.
Stage 2: Model diffing
Training an SAE directly on a multimodal model yields a tangle of features inherited from the language model mixed with newly added multimodal features. MMDiff instead compares two SAEs:
1. An SAE on the base language model (text only) 2. An SAE adapted to multimodal data (text + images)
By contrasting them, it identifies features 'rewritten' by multimodal training.
Stage 3: Feature-level control
Technical pipeline
1. Multimodal SAE training: continue training the base LM's SAE on multimodal data while keeping feature indices aligned. 2. Adapted-feature identification via two signals:
Results
Validated on three MLLM families (LLaVA-MORE, PaliGemma 2, InternVL3.5):
| Task | Causal removal | Steering gain | |------|----------------|---------------| | Spatial reasoning | -12% | +3.6% | | OCR | -17% | +1.8% | | Multimodal safety | ASR reduced by 24% | — |
Key finding: removing these features barely affects general VQA ability (|ΔVQA| ≤ 1.5%), demonstrating the specificity of the identified features.
Closing remark from the original post
> 'MMDiff is like a neuro-archaeologist digging through the ruins of Transformer layers—not looking for a whole civilization, but for the neuron fingerprints rewritten by multimodal training. Each identified feature is a window into the AI's inner world.'
*Original interpretation by Xiao Kai, Feynman-style deep dive.*