Summary
This paper (arXiv:2609.26986, Zhuoyun Li, Boxuan Wang, Xiaowei Huang, Yi Dong) studies how evidence order affects judgments of multimodal large language models when images or speech conflict with accompanying text. Earlier studies of text bias often used a fixed evidence order or moved task instructions along with the evidence, leaving the contribution of order unclear. The authors introduce a paired comparison that keeps instructions and evidence content fixed while swapping only the positions of the two sources. Across vision and speech models, placing an image or recording after conflicting text consistently shifts answers toward its content. The paper also revisits prior studies and analyzes why their experimental settings may lead to misleading conclusions. The findings reveal that cross-modal evidence is non-commutative: the same evidence can yield different judgments depending on order, and placing perceptual evidence later increases the model's reliance on it.
Paper Overview
Field: Machine Learning
Authors: Zhuoyun Li, Boxuan Wang, Xiaowei Huang, Yi Dong
Published: 2026-09-25
arXiv: 2609.26986
Abstract
For multimodal large language models, when images or speech conflict with accompanying text, measured text reliance can entangle modality preference with evidence position. Earlier studies of text bias often used a fixed evidence order or moved task instructions with the evidence, leaving the contribution of order unclear. In this paper, we use a paired comparison that keeps the instructions and evidence content fixed and swaps only the positions of the two sources to quantify this potential influence. Across vision and speech models, placing an image or recording after conflicting text consistently shifts answers toward its content. We also revisit previous studies and analyze why their experimental settings can lead to misleading conclusions. These findings reveal cross-modal evidence noncommutativity: the same evidence can lead to different judgments when the order changes; placing perceptual evidence later strengthens the model's reliance on its content.
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178635176