[论文] From Static to Dynamic: Benchmarking Real-World Code Review with MCR-B...

论文概要

研究领域: NLP 作者: Dewu Zheng, Yanlin Wang, Xiwen Wang 发布时间: 2026-08-28 arXiv: 2508.11368

中文摘要

在真实软件开发中，代码审查通常涉及开发者与审查者之间的迭代交互以提升软件质量，这使得该过程既昂贵又耗时。尽管近期研究探索了大语言模型（LLMs）用于自动代码审查，但大多数方法将代码审查过度简化为单轮静态决策任务，未能捕捉真实审查场景中固有的多轮交互性质和复杂问题解决过程。为弥合这一差距，我们引入MCR-Bench——首个针对真实多轮代码审查的缺陷状态感知基准。MCR-Bench涵盖五种常用编程语言，包含2,269个真实多轮代码审查任务，每个任务都标注了细粒度缺陷信息和跨轮状态标签。MCR-Bench中的每个任务配备细粒度缺陷元数据（如描述、类型、严重度）以及动态状态注释，捕捉缺陷在多轮过程中的完整演化轨迹。通过与主流LLM在MCR-Bench上的大量实验，我们获得几个发现：(1) 整体能力有限：实验表明主流LLM在缺陷检测和缺陷生命周期状态跟踪方面整体性能有限，且随着交互轮数增加性能显著下降；(2) 缺陷敏感性能：LLM在不同缺陷类型和严重级别间性能差异显著，语义复杂或低显著性缺陷更可能被遗漏；(3) 底层失败机制：我们的深入错误分析剖析了假阳性和假阴性的不同驱动因素，揭示了关键弱点如跨轮时间错位和远程记忆不足。

原文摘要

In real-world software development, code review typically involves iterative interactions between developers and reviewers to improve software quality, making the process costly and time-consuming. Although recent work explores large language models (LLMs) for automated code review, most approaches oversimplify code review into a single-round, static decision task, which fails to capture the multi-round interactive nature and the complex problem-solving processes inherent in realistic review scenarios. To bridge this gap, we introduce MCR-Bench, the first defect state-aware benchmark designed for realistic multi-round code review. MCR-Bench covers five commonly-used programming languages and consists of 2,269 real-world multi-round code review tasks, each of which is annotated with fine-gr...

#论文 #arXiv #NLP #小凯