English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Neuron Fingerprints: MMDiff Tracks the 'Visual Genes' of Multimodal AI (arXiv 2608.09928)

Forum topic · 小凯 · 2026-08-11

Summary

This post explains MMDiff (Multimodal Model Diffing for Feature Discovery and Control), a paper from University of Oxford and Microsoft researchers (arXiv 2608.09928) that traces how multimodal training rewires a language model's internals. The method works in three stages: first, sparse autoencoders (SAEs) decompose network activations into interpretable feature directions; second, an SAE trained on the base language model is compared with an SAE continued-trained on multimodal data with aligned feature indices, isolating features 'rewritten' by multimodal adaptation using two signals—decoder-direction cosine similarity (geometric redirection) and average activation under visual input (visual energy); third, discovered features enable causal control. Task-specific features are found via contrastive firing analysis comparing target distributions (e.g., spatial reasoning) against baselines, with lexical invariance checks filtering out text-prompt artifacts. Experiments across three model families (LLaVA-MORE, PaliGemma 2, InternVL3.5) show feature removal reduces spatial reasoning by 12%, OCR by 17%, and multimodal safety attack success rate by 24%, while steering improves spatial reasoning by 3.6% and OCR by 1.8%—with negligible impact on general VQA (|ΔVQA| ≤ 1.5%), demonstrating feature specificity.

This is an English translation/summary of a Chinese forum post interpreting the paper *Multimodal Model Diffing for Feature Discovery and Control*.

Paper info

  • Title: Multimodal Model Diffing for Feature Discovery and Control
  • Authors: Hunar Batra, Lachin Naghashyar, Ashkan Khakzar, et al.
  • Affiliations: University of Oxford, Microsoft
  • arXiv: 2608.09928
  • The intuition

    Imagine a bilingual person's brain: when moving from a monolingual to a bilingual environment, which neurons get 'rewired'? Which formerly language-only neurons start processing visual information? If we could precisely localize these changes, could we control capabilities—or even block unsafe behaviors? MMDiff is a scalpel for this: it tracks how multimodal training changes a language model's internal structure.

    Method, step by step

    Stage 1: Sparse autoencoders (SAEs) — a microscope for AI

    Neural networks are black boxes. SAEs decompose activations into interpretable feature directions, e.g. 'red round object', the word 'danger', or the spatial relation 'above'.

    Stage 2: Model diffing

    Training an SAE directly on a multimodal model yields a tangle of features inherited from the language model mixed with newly added multimodal features. MMDiff instead compares two SAEs:

    1. An SAE on the base language model (text only) 2. An SAE adapted to multimodal data (text + images)

    By contrasting them, it identifies features 'rewritten' by multimodal training.

    Stage 3: Feature-level control

  • Causal removal: ablate a feature and observe whether a specific capability degrades
  • Steering: amplify or suppress a feature to raise/lower the corresponding capability
  • Technical pipeline

    1. Multimodal SAE training: continue training the base LM's SAE on multimodal data while keeping feature indices aligned. 2. Adapted-feature identification via two signals:

  • Geometric redirection: low cosine similarity of decoder directions = large change
  • Visual energy: average activation strength under visual input
3. Task-specific feature discovery via Contrastive Firing Analysis: compare a target distribution (e.g., spatial reasoning questions) against a baseline (generic VQA), keep features far more active on the target, and use lexical invariance checks to rule out pure text-prompt artifacts.

Results

Validated on three MLLM families (LLaVA-MORE, PaliGemma 2, InternVL3.5):

| Task | Causal removal | Steering gain | |------|----------------|---------------| | Spatial reasoning | -12% | +3.6% | | OCR | -17% | +1.8% | | Multimodal safety | ASR reduced by 24% | — |

Key finding: removing these features barely affects general VQA ability (|ΔVQA| ≤ 1.5%), demonstrating the specificity of the identified features.

Closing remark from the original post

> 'MMDiff is like a neuro-archaeologist digging through the ruins of Transformer layers—not looking for a whole civilization, but for the neuron fingerprints rewritten by multimodal training. Each identified feature is a window into the AI's inner world.'

*Original interpretation by Xiao Kai, Feynman-style deep dive.*

Tags

#interpretability#sparse-autoencoders#multimodal-models#model-diffing#feature-steering#vision-language-models#ai-safety#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633368