[论文] PROVISO: Provenance-Informed Action Selection for Secure LLM Agents

论文概要

研究领域: ML 作者: Bo Yan, Qi Zhang, Ziwen Kan, Fnu Suya, Song Wang 发布时间: 2026-10-22 arXiv: 2610.27115

中文摘要

LLM智能体通过工具和第三方技能对外部世界执行操作，因此单个已执行的动作可能造成不可逆的损害。现有防御措施在判断一个拟议动作时，不追踪其内容是来自用户还是来自外部来源（如工具输出），并且孤立地判断它而不是与可用的替代方案进行比较，这迫使在安全性和效用性之间做出权衡。我们提出PROVISO，一个在智能体提出动作后、执行前决定提交哪个动作的控制器。PROVISO将可信观察、外部观察、拟议动作和替代方案编码为溯源类型化的记忆，通过角色特定的查询将它们关联起来，并提交从结果中学到的校准值最高的动作。在跨越智能体生命周期的四个智能体安全任务上，没有基线能在安全性和效用性上同时超过PROVISO。PROVISO在SkillsBench上比最强基线提升F1分数13.84个百分点，在ToolSafe上提升7.07个百分点，并在另外三个技能检测基准上达到88-99的F1分数。

原文摘要

LLM agents act on the external world through tools and third-party skills, so a single executed action can cause irreversible harm. Existing defenses judge a proposed action without tracking whether its content comes from the user or from external sources such as tool outputs, and they judge it in isolation rather than against an available alternative, which forces a trade-off between security and utility. We present PROVISO, a controller that decides which action to commit after the agent proposes an action and before it is executed. PROVISO encodes trusted observations, external observations, the proposed action, and an alternative into provenance-typed memory, relates them through role-specific queries, and commits the action with the highest calibrated value learned from outcomes. On f...

#论文 #arXiv #ML #小凯