[论文] FrontierPhysics: Evaluating Agents for End-to-End Physics Research

论文概要

研究领域: ML 作者: Bingran You, Wenjun Ke, Ria Rosenauer, Bo Liu, Wenbo Chen, Min Chen, Shuqi Xu, Yue Ma, Jiechao Feng, Bo Chen, Xiangkai Sun, Jiarui Liu, Jinen Guo, Zeyu Zhou, Xu Huang, Zhuohan Li, Haoyu Li, Heguang Lin, Lijie Ding, Yadong Zeng, Kangqi Zhang, Xinge Liu, Hanwen Liu, Zhenghong Li, Alexander von Recum, Yuchen Zhu, Felix Schwarzfischer, Yizhou Zhao, Jicheng Wang, Binxu Li, Qian Yu, Chan Li, Cheng Dai, Di Wang, Haoyu Zhang, Jiankai Sun, Kaituo Feng, Mian Qin, Ruihong Yin, Yiqun Luo, Weiwei Wu, Shihong Pan, Tsung Yeh Hsieh, Xiao Huang, Yan Zhang, Sihan Li, Zirui Ren, Zengji Tu, Kyoung Whan Choe, Ke Sun, Vincenzo Galia, Vaisakhi Mishra, Jiashu He, Hongyang Wang, Zhiheng Yang, Shubham Gaur, Yifan Gu, Shuyi Wang, Zihao Zuo, Steven Dillmann, Yiyou Sun, Xinyang Han, Hanchung Lee, Molei Tao, Yue Zhao, Shilong Liu, Yuchen Wu, Xuandong Zhao, Xiangyi Li, Florian Brand, Gerbrand Ceder, Hartmut Haeffner 发布时间: 2026-11-04 arXiv: 2611.06910

中文摘要

AI 智能体在物理研究中正扮演越来越重要的角色。然而现有物理基准仅评估固定问题、孤立子任务或给定解决方案的论文复现，很少同时检验研究质量和正确性。我们提出 FrontierPhysics——一个端到端基准，包含 42 位物理学家基于自身研究构建的 56 项任务，涵盖理论、计算和实验物理九个领域。每项任务要求智能体完成文献综述、制定研究计划、开展研究和实验，并产出结构化结果和科学论文。结合确定性验证器和专家编写的评分标准，FrontierPhysics 评估智能体的研究是否正确且具备发表水准。在 22 个模型 26 种配置中，没有任何配置通过其 168 次运行中超过 46 次。对智能体轨迹的人工审查表明，前沿智能体的主要弱点在于推理不够严谨和专家直觉有限，而非物理知识缺口或执行错误。

原文摘要

AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26...

#论文 #arXiv #ML #小凯