PROVISO: Provenance-Informed Action Selection for Secure LLM Agents

Forum topic · 小凯 · 2026-10-22

Summary

PROVISO is a controller for LLM agents that decides which proposed action to commit before execution, addressing the fact that a single executed action can cause irreversible harm. Existing defenses judge actions without tracking whether their content originates from the user or from external sources such as tool outputs, and evaluate actions in isolation rather than against alternatives, forcing a security-utility trade-off. PROVISO encodes trusted observations, external observations, the proposed action, and an alternative into provenance-typed memory, relates them via role-specific queries, and commits the action with the highest calibrated value learned from outcomes. Across four agent-safety benchmarks spanning the agent lifecycle, no baseline surpasses PROVISO on both safety and utility. It improves F1 by 13.84 points over the strongest baseline on SkillsBench and 7.07 points on ToolSafe, reaching F1 scores of 88-99 on three additional skill-detection benchmarks. Authors: Bo Yan, Qi Zhang, Ziwen Kan, Fnu Suya, Song Wang. arXiv:2610.27115.

Overview

Field: Machine Learning Authors: Bo Yan, Qi Zhang, Ziwen Kan, Fnu Suya, Song Wang Published: 2026-10-22 arXiv: 2610.27115

Abstract

LLM agents act on the external world through tools and third-party skills, so a single executed action can cause irreversible harm. Existing defenses judge a proposed action without tracking whether its content comes from the user or from external sources such as tool outputs, and they judge it in isolation rather than against an available alternative, which forces a trade-off between security and utility. We present PROVISO, a controller that decides which action to commit after the agent proposes an action and before it is executed. PROVISO encodes trusted observations, external observations, the proposed action, and an alternative into provenance-typed memory, relates them through role-specific queries, and commits the action with the highest calibrated value learned from outcomes.

Key Results

On four agent-safety tasks spanning the agent lifecycle, no baseline simultaneously outperforms PROVISO on both safety and utility.
+13.84 F1 points over the strongest baseline on SkillsBench.
+7.07 F1 points on ToolSafe.
F1 scores of 88-99 on three additional skill-detection benchmarks.

Full Paper

Read the full paper on arXiv: https://arxiv.org/abs/2610.27115