Neuron Fingerprints: Tracking the 'Visual Genes' Acquired by Multimodal AI

Forum topic · 小凯 · 2026-08-11

Summary

A research commentary on 'Multimodal Model Diffing for Feature Discovery and Control' (arXiv:2608.09928) by Batra et al. from the University of Oxford and Microsoft. The paper introduces MMDiff, a method for identifying which internal features of a language model are rewritten when it is adapted into a multimodal large language model (MLLM). MMDiff compares sparse autoencoder (SAE) feature dictionaries between a base text-only model and its multimodal counterpart, then detects adapted features via geometric redirection (low cosine similarity of decoder directions) and visual energy (activation on image inputs). A contrastive firing analysis isolates task-specific features. Experiments on LLaVA-MORE, PaliGemma 2, and InternVL3.5 show that causal removal of these features degrades spatial reasoning by 12%, OCR by 17%, and multimodal safety attack success rate by 24%, while steering yields modest gains (+3.6% spatial, +1.8% OCR). General VQA accuracy changes by at most 1.5%, confirming feature specificity.

Paper Information

Original title: Multimodal Model Diffing for Feature Discovery and Control
Authors: Hunar Batra, Lachin Naghashyar, Ashkan Khakzar, et al.
Institutions: University of Oxford, Microsoft
arXiv: 2608.09928

Intuitive Analogy

Imagine a bilingual brain. When it switches from monolingual to bilingual processing, which neurons get rewired? Which language-only neurons begin handling visual information? If we can precisely locate these changes, can we control language ability, visual understanding, or even suppress unsafe thoughts?

MMDiff acts as a "dissection scalpel" that traces how multimodal training reshapes the internal structure of a language model.

Step-by-Step Method

Stage 1: Sparse Autoencoders (SAE) as a Microscope

Neural networks are black boxes. An SAE works like a microscope, decomposing activations into interpretable directions such as "red circular object," the word "danger," or the spatial relation "above."

Stage 2: Model Diffing

Training an SAE directly on a multimodal model yields a tangled mixture of inherited language features and newly added multimodal features. MMDiff instead compares two aligned SAEs:

1. A base text-only SAE 2. A multimodal-adapted SAE

By diffing the two, the method isolates features that have been rewritten by multimodal training.

Stage 3: Feature-Level Control

Once located, MMDiff supports two interventions:

Causal Removal: Delete a target feature and measure capability loss
Steering: Amplify or suppress a feature to enhance or weaken the corresponding ability

Technical Pipeline

1. Multimodal SAE training. Initialize from the base LLM's SAE, continue training on multimodal data while keeping feature indices aligned. 2. Adapted feature identification via two signals:

Geometric redirection: cosine similarity between decoder directions (low similarity = large change)
Visual energy: mean activation magnitude on visual inputs

Task-specific feature discovery

Compare a target distribution (e.g., spatial reasoning) against a baseline (generic VQA)
Filter features that fire significantly more on the target distribution
Apply a lexical-invariance check to rule out text-only prompt artifacts

Experimental Results

Validated across three MLLM families: LLaVA-MORE, PaliGemma 2, and InternVL3.5.

| Task | Causal Removal Effect | Steering Gain | |------|----------------------|---------------| | Spatial reasoning | -12% | +3.6% | | OCR | -17% | +1.8% | | Multimodal safety | Attack success rate -24% | — |

A key finding: removing these features leaves general VQA accuracy nearly untouched (|ΔVQA| ≤ 1.5%), confirming that the identified features are specific to the targeted capability rather than general-purpose.

Significance

MMDiff reframes multimodal alignment as a feature-rewriting problem. Instead of treating the multimodal model as an opaque whole, it provides a principled, neuron-level audit of what visual training actually changes inside the language backbone. This makes it a useful diagnostic tool for safety auditing, capability localization, and controlled editing of MLLMs.