Dude: A Dual-Detection Multi-Agent System for Paper-Code Discrepancy Detection

Forum topic · 小凯 · 2026-09-06

Summary

Dude is a dual-detection multi-agent system built on LLMs to detect discrepancies between research papers and their accompanying code, addressing the gap as research submission volume outgrows manual review capacity. The authors identify two limitations of single-agent LLM approaches: limited context capacity and one-sided discrepancy detection, both leading to poor recall. They also find that the granularity asymmetry between paper language and code language causes over-interpretation and over-reporting in multi-agent designs, increasing false positives. To mitigate this, Dude introduces a granularity-aligned negotiation mechanism and a two-stage salience-filtering mechanism that prevent agents from incorrectly reporting discrepancies. Experiments on a real paper-code discrepancy dataset show Dude improves recall and precision by up to 22.8% and achieves F1 scores up to 18.7% higher than baseline methods. The paper is available at arXiv:2509.00007.

Overview

Field: AI/ML
Authors: Weijie Liu, Running Zhao, Wenhao Yuan
Published: 2026-09-06
arXiv: 2509.00007

Abstract (translated)

As the scaling of research submissions exceeds manual review capability, LLM-empowered paper-code discrepancy detection has received growing attention. However, the limited context capacity and one-sided discrepancy detection of existing single-agent LLM paradigms lead to inferior recall performance.

This paper proposes Dude, the first Dual-Detection Multi-Agent System for paper-code discrepancy detection. The authors discover that the granularity asymmetry between paper language and code language introduces over-interpretation and over-reporting challenges in multi-agent system designs for discrepancy detection, resulting in increased false positives.

Key Contributions

Dual-detection multi-agent architecture: Dude is the first system to apply a dual-detection multi-agent design to paper-code discrepancy detection, addressing the recall limitations of single-agent LLM approaches.
Granularity-aligned negotiation: A mechanism to reconcile the granularity mismatch between paper text and source code, preventing agents from misinterpreting or misreporting discrepancies.
Two-stage salience filtering: A filtering pipeline that reduces false positives caused by over-reporting.

Results

Experiments on a real-world paper-code discrepancy dataset show that Dude significantly improves recall and precision by up to 22.8%, and achieves F1 scores up to 18.7% higher than baseline methods.

Original Abstract

> LLM-empowered paper-code discrepancy detection has received growing concern since the scaling of research submissions exceeds the manual review capability. However, the limited context capacity and one-sided discrepancy detection of existing single-agent LLM paradigms lead to an inferior recall performance in detecting discrepancies. In this paper, we propose Dude, the first Dual-Detection Multi-Agent System for paper-code discrepancy detection. We discover that the granularity asymmetry of the paper-language and code-language introduces over-interpretation and over-reporting challenges in a multi-agent system design for discrepancy detection, resulting in increasing false positives. To address this, we propose a granularity-aligned negotiation and a two-stage salience-filtering mechanism ...

---

*Auto-collected on 2026-09-06*