[论文] PRIME: Towards Verifiable Evidence-Grounded Reasoning in LLM Agents

论文概要

研究领域: NLP 作者: Hamed Khosravi, Xiaoming Huo 发布时间: 2026-10-19 arXiv: 2610.22426

中文摘要

LLM 智能体常被用来核对记录是否满足组织要求（如漏洞报告是否达到可调查标准），而要求往往允许多条满足路径——如「评级+经策展的咨询公告」或「已发布公告链接」皆可。智能体的典型失败模式：某项必要检查已失败仍接受报告，或未尝试仍可成功的更强路径就满足于较弱结论。PRIME（基于最小证据的预承诺推理规则）在检查前固定证据路由，规定每个结论可用哪些方式证明；一个独立程序记录每项检查结果，拒绝结果不足以支撑的结论，并指引（或代为执行）能支撑更强结论的检查。PRIME 只报告规则与账本授权的结论。在 480 个公开记录任务上：直接看要求的智能体在 26.7% 的任务上报告了无依据结论，PRIME 为零；其指引与补充检查把有依据结论的平均分从仅拒绝时的 30.1 提到最高 59.2（百分制）。在要求正确、检查位置已知、预算充足的前提下，完成时可达到最强可支撑结论。

原文摘要

LLM agents are used to check records against an organization's requirements, for example whether a vulnerability report is ready for investigation. Requirements often allow several ways to be met, such as a severity rating with a curator-reviewed advisory or a published advisory link. Agents fail by accepting a report after a required check has failed, or by settling for a weaker conclusion without trying another way that could still succeed. We introduce PRIME (Precommitted Rules for Inference from Minimal Evidence). Before checking, it fixes evidence routes specifying how each conclusion may be justified. A separate program records each check result and refuses any conclusion the results do not justify. It also points the agent to checks that could justify a stronger conclusion, or runs them itself. PRIME reports only conclusions authorized by its rules and ledger. With correct requirements, exact checks at known locations, and sufficient budget at stopping, completion reaches the strongest supported conclusion. On 480 public-record tasks, an agent shown the requirements reported unjustified conclusions on 26.7% of tasks, while PRIME reported none. Its directions and added checks raised the average score for justified conclusions from 30.1, with refusal alone, to the maximum of 59.2 out of 100.

---

*自动采集于 2026-10-19*

#论文 #arXiv #NLP #小凯