[论文] AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Pr...

论文概要

研究领域: NLP 作者: Lingkai Kong, Zijian Wu, Yuzhe Gu, Haiteng Zhao, Wenyong Huang 发布时间: 2026-07-13 arXiv: 2607.11849

中文摘要

大语言模型在高中和奥赛风格数学上取得了显著成绩，但其在高等数学上的能力仍知之甚少。现有基准在范围和评估粒度上均有不足：学科覆盖有限，且常依赖最终答案正确性或粗粒度判断，对推理过程的有效性评估不充分。本文提出AdvancedMathBench，用于评估高等数学推理能力的基准套件。其核心证明生成基准ProverBench包含296道涵盖本科和博士资格考试难度的问题。为可靠评估证明，开发了基于大规模专家标注训练的自动验证流水线，输出正确性判决和证明错误的细粒度评估，在保留证明轨迹上与人类专家高度一致。还引入VerifierBench，包含888条模型生成证明轨迹与专家真值配对，评估模型是否能正确判断证明有效性并提供合理验证理由。实验表明AdvancedMathBench对前沿模型仍具挑战性：最佳证明生成模型GPT-5.5-xhigh在UGD和QE划分上仅达75.8和66.1；证明验证上最佳模型Balanced F1仅65.1，真阴性率低，关键错误检测仍是主要瓶颈。

原文摘要

Large language models (LLMs) have achieved remarkable performance on high-school and olympiad-style mathematics, yet their capabilities on advanced mathematics remain poorly understood. Existing benchmarks, however, fall short in both scope and evaluation granularity: they provide limited disciplinary coverage and often rely on final-answer correctness or coarse judgments, leaving the validity of the reasoning process inadequately assessed. To bridge this gap, we introduce AdvancedMathBench, a benchmark suite designed to evaluate advanced mathematical reasoning capabilities. Its core proof-generation benchmark, ProverBench, contains 296 problems spanning undergraduate and doctoral qualifying-exam levels. To provide reliable evaluation of the proofs, we develop a dedicated automatic verific...

--- *自动采集于 2026-07-15*

#论文 #arXiv #NLP #小凯