DeepSciVerify: Two-Stage Pipeline for Verifying Scientific Claim–Citation Alignment

Forum topic · 小凯 · 2026-05-29

Summary

DeepSciVerify is a two-stage pipeline for verifying alignment between scientific claims and their cited evidence, addressing a common failure mode in reports generated by large language models. The system first verifies claims using only paper abstracts and defers uncertain cases, escalating to full-text passage retrieval and analysis only when necessary. This design exploits complementary behaviors across LLMs, since some models are more conservative while others are more decisive under uncertainty. On the SCitance benchmark, DeepSciVerify achieves 86.7 Micro-F1, outperforming strong abstract-only baselines by +4.5 points while resolving 67% of instances without full-text retrieval. The results indicate that selective evidence escalation improves both the accuracy and efficiency of claim-citation verification. Paper by Shaghayegh Sadeghi, Khashayar Khajavi, Rise Adhikari, et al., available on arXiv (2605.27710).

Overview

Misalignment between claims and their cited evidence is a common failure mode in reports generated by large language models, limiting their reliability in scientific and other high-stakes settings. This post introduces DeepSciVerify, a two-stage pipeline for scientific claim-citation verification that combines abstract-level reasoning with selective escalation to passage-level evidence.

How It Works

The system first verifies claims using the paper's abstract.
Uncertain cases are deferred rather than forced to a decision.
Full-text passages are retrieved and analyzed only when necessary.
The design leverages complementary behaviors across LLMs: some models are more conservative, while others are more decisive under uncertainty.

Results

On the SCitance benchmark, DeepSciVerify achieves 86.7 Micro-F1.
This outperforms strong abstract-only baselines by +4.5 points.
67% of instances are resolved without full-text retrieval, improving efficiency.

Conclusion

Selective evidence escalation can simultaneously improve the accuracy and efficiency of claim-citation verification, suggesting a practical approach for more reliable LLM-generated scientific reports.

Paper: Shaghayegh Sadeghi, Khashayar Khajavi, Rise Adhikari, et al. — arXiv:2605.27710