[论文] MAMath-Bench: Process-Aware and Robust Evaluation of Multimodal Multi-...

论文概要

研究领域: ML
作者: Xinping Wang, Jie Ma, Rui Xing, Wensong Guo, Xiayi Zhang, Jianhao Deng, Zihao Zhou, Boqian Yang, Feijuan He
发布时间: 2026-10-22
arXiv: 2610.26943

中文摘要

数学推理是大语言模型（LLM）教育应用的基础能力，可靠的问题求解不仅需要正确答案，还需要连贯且可验证的推理过程。多智能体系统（MAS）通过利用多个LLM智能体之间的协作和验证，在该领域展现出日益增长的潜力。然而，现有评估主要集中在纯文本设置下最终答案的正确性，对多模态推理能力、中间推理步骤的可靠性以及对抗推理错误的鲁棒性提供了有限的洞察。为弥补这些不足，我们引入MAMath-Bench，一个面向MAS多模态数学推理能力的过程感知评估基准。MAMath-Bench包含1003个问题，涵盖11个数学类别、多个难度级别和多种模态（包括文本、图像和音频）。它采用细粒度的里程碑标注，从三个维度评估系统：问题求解正确性、推理过程可靠性和对抗推理链扰动的鲁棒性。我们的评估得出三个关键发现：首先，多智能体协作并不能持续提升最终答案准确率以优于强大的单LLM；其次，协作可以显著增强中间推理的覆盖率，但这种增益可能以增加推理冗余为代价；第三，当前系统对对抗性推理错误仍然脆弱，尤其是涉及细微结构性扰动的错误。

原文摘要

Mathematical reasoning is a fundamental capability for educational applications of Large Language Models (LLMs), where reliable problem solving requires not only correct answers but also coherent and verifiable reasoning processes. Multi-agent systems (MASs) have shown growing potential in this domain by leveraging collaboration and verification among multiple LLM agents. However, existing evaluations primarily focus on the correctness of final answers in text-only settings, offering limited insight into multimodal reasoning capabilities, the reliability of intermediate reasoning steps, and robustness against reasoning errors. To address these limitations, we introduce MAMath-Bench, a process-aware benchmark designed to evaluate the multimodal mathematical reasoning capabilities of MASs. M...

自动采集于 2026-10-22

#论文 #arXiv #ML #小凯