[论文] From Static to Dynamic: Benchmarking Real-World Code Review with MCR-B...

论文概要

研究领域: NLP 作者: Dewu Zheng, Yanlin Wang, Xiwen Wang, Kefeng Duan, Hongyu Zhang, Xilin Liu, Yuchi Ma, Zibin Zheng 发布时间: 2026-08-27 arXiv: 2608.27442

中文摘要

在现实软件开发中，代码审查通常涉及开发者和审查者之间的迭代交互以提升软件质量，使这一过程成本高昂且耗时。尽管近期研究探索了大语言模型（LLMs）用于自动化代码审查，但大多数方法将代码审查过度简化为单轮静态决策任务，未能捕捉现实审查场景中的多轮交互性质和复杂问题解决过程。为弥合这一差距，我们引入了MCR-Bench，首个针对真实多轮代码审查的缺陷状态感知基准。MCR-Bench涵盖五种常用编程语言，包含2,269个真实多轮代码审查任务，每个任务都标注了细粒度缺陷信息和跨轮状态标签。通过主流LLM的广泛实验，我们发现：（1）整体能力有限：性能随交互轮数增加显著下降；（2）缺陷敏感性能：LLM在不同缺陷类型和严重程度上的性能差异显著；（3）底层失败机制：错误分析揭示了跨轮时间错位和长程记忆不足等关键弱点。

原文摘要

In real-world software development, code review typically involves iterative interactions between developers and reviewers to improve software quality, making the process costly and time-consuming. Although recent work explores large language models (LLMs) for automated code review, most approaches oversimplify code review into a single-round, static decision task, which fails to capture the multi-round interactive nature and the complex problem-solving processes inherent in realistic review scenarios. To bridge this gap, we introduce MCR-Bench, the first defect state-aware benchmark designed for realistic multi-round code review. MCR-Bench covers five commonly-used programming languages and consists of 2,269 real-world multi-round code review tasks, each of which is annotated with fine-gr...

#论文 #arXiv #NLP #小凯