MAMath-Bench: A Process-Aware Benchmark for Multimodal Math Reasoning in Multi-Agent LLM Systems

Forum topic · 小凯 · 2026-10-22

Summary

MAMath-Bench is a process-aware benchmark for evaluating multimodal mathematical reasoning in multi-agent systems (MAS) built on large language models (LLMs). The benchmark contains 1003 problems spanning 11 math categories, multiple difficulty levels, and multiple modalities including text, images, and audio. It uses fine-grained milestone annotations to assess systems along three dimensions: final answer correctness, reliability of the reasoning process, and robustness against adversarial perturbations of reasoning chains. Evaluation results reveal three key findings: first, multi-agent collaboration does not consistently improve final answer accuracy over strong single LLMs; second, collaboration substantially increases intermediate reasoning coverage, but this gain may come at the cost of greater reasoning redundancy; third, current systems remain vulnerable to adversarial reasoning errors, particularly subtle structural perturbations. The paper (arXiv:2610.26943) addresses the gap left by prior evaluations that focus mainly on final-answer correctness in text-only settings, offering deeper insight into multimodal reasoning quality and robustness for educational applications of LLMs.

Paper Overview

Field: Machine Learning Authors: Xinping Wang, Jie Ma, Rui Xing, Wensong Guo, Xiayi Zhang, Jianhao Deng, Zihao Zhou, Boqian Yang, Feijuan He Published: 2026-10-22 arXiv: 2610.26943

Summary

Mathematical reasoning is a fundamental capability for educational applications of Large Language Models (LLMs), where reliable problem solving requires not only correct answers but also coherent and verifiable reasoning processes. Multi-agent systems (MASs) have shown growing potential in this domain by leveraging collaboration and verification among multiple LLM agents. However, existing evaluations primarily focus on the correctness of final answers in text-only settings, offering limited insight into multimodal reasoning capabilities, the reliability of intermediate reasoning steps, and robustness against reasoning errors.

To address these limitations, the authors introduce MAMath-Bench, a process-aware benchmark designed to evaluate the multimodal mathematical reasoning capabilities of MASs.

Benchmark Design

1003 problems covering 11 mathematical categories
Multiple difficulty levels and multiple modalities (text, images, and audio)
Fine-grained milestone annotations
Three evaluation dimensions:

1. Problem-solving correctness 2. Reasoning-process reliability 3. Robustness against adversarial perturbations of reasoning chains

Key Findings

1. Multi-agent collaboration does not consistently improve final answer accuracy over strong single LLMs. 2. Collaboration significantly enhances intermediate reasoning coverage, but this gain may come at the cost of increased reasoning redundancy. 3. Current systems remain fragile against adversarial reasoning errors, especially those involving subtle structural perturbations.

---

*Auto-collected on 2026-10-22*

MAMath-Bench: A Process-Aware Benchmark for Multimodal Math Reasoning in Multi-Agent LLM Systems

Summary

Paper Overview

Summary

Benchmark Design

Key Findings

Tags