[论文] How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of...

论文概要

研究领域: NLP
作者: Miguel Mochizuki Silva, Samuel Ismael, Mateus C. Dias, Thaís Gaudêncio, Yuri Malheiros, Barufaldi
发布时间: 2026-10-22
arXiv: 2610.31606

中文摘要

目的：为乳腺病变管理文献的临床RAG系统描述循证感知的检索评估协议。方法：使用18篇文档语料库和18个查询基准，检索结合密集嵌入、BM25和交叉编码器重排序，评估128-1024令牌块大小、0-20%重叠率和四种检索器配置。结果：选定384令牌块+20%重叠的混合检索器，18个查询中17个在前三结果检索到可接受证据，Recall@3、R-Precision、MRR@3、nDCG@3均达0.94。真实值整理将证据跨度从25增至37但未改变检索排名。结论：交叉编码器重排序获得最高检索分数，扩展相关性标注在不改变排名的情况下增加了识别的证据，凸显标注敏感性。

原文摘要

Purpose: To describe an evidence-aware retrieval evaluation protocol for a clinical RAG system over breast lesion management literature. Methods: An 18-document corpus and 18-query development benchmark were used. Retrieval combined dense embeddings, BM25, and cross-encoder reranking. Chunk sizes of 128 to 1024 tokens, overlaps of 0 to 20%, and four retriever configurations were evaluated. Results: The selected hybrid retriever, using 384-token chunks and 20% overlap, retrieved accepted evidence in the top three results for 17 of 18 queries. Recall@3, R-Precision, MRR@3, and nDCG@3 each reached 0.94. Ground-truth curation increased evidence spans from 25 to 37 without changing retrieval rankings. Conclusion: Cross-encoder reranking achieved the highest retrieval scores. Expanded relevance ...

自动采集于 2026-10-26

#论文 #arXiv #NLP #小凯