[论文] Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via P...

论文概要

研究领域: CV
作者: Seojeong Park, Jiho Choi, Junyong Kang
发布时间: 2026-06-03
arXiv: 2506.00002

中文摘要

近期的多模态大语言模型已展现出强大的推理能力，但它们作为自动化评估器的可靠性仍存在一个关键弱点：当视觉证据与文本线索冲突时，MLLM评估器倾向于奖励看似合理的叙述而非感知上正确的答案。我们识别并系统分析了这一现象，将其称为'感知判断偏差'。通过受控的视觉扰动实验，我们发现现有的多模态评估器频繁地锚定在回应文本上，而非依赖自身的视觉感知，导致不一致且无法验证的评估结果。为解决这个问题，我们引入了感知扰动判断数据集，该数据集构建最小编辑的反事实响应，以隔离感知错误并实现可验证的监督。基于该数据集，我们开发了一个统一的训练框架，结合结构化的GRPO奖励与批次排序目标，在没有显式成对标签的情况下实现连贯的全局排序。在多种MLLM-as-a-Judge基准测试上的实验表明，我们的方法显著提升了感知保真度、排序一致性以及与人类评估的对齐度。我们的结果为训练具有感知基础、可解释且对视觉-推理冲突具有鲁棒性的多模态评估器建立了一条可扩展且可泛化的路径。

原文摘要

Recent multimodal large language models have demonstrated strong reasoning ability, yet their reliability as automated evaluators remains limited by a critical weakness: when visual evidence conflicts with textual cues, MLLM judges tend to reward plausible narratives over perceptually correct answers. We identify and systematically analyze this phenomenon, which we term Perceptual Judgment Bias. Through controlled visual perturbations, existing multimodal judges frequently anchor on the response text instead of their own visual perception, leading to inconsistent and non-verifiable evaluations. To address this issue, we introduce the Perceptually Perturbed Judgment Dataset, which constructs minimally edited counterfactual responses that isolate perceptual errors and enable verifiable super...

自动采集于 2026-06-03

#论文 #arXiv #CV #小凯