[论文] [论文] I-HITL: Inalienable Human-in-the-Loop

论文概要

研究领域: ML
作者: Samuel Breckenridge, Dani Vilardell, Wenhao Wang, Fan Zhang, Ari Juels
发布时间: 2026-10-26
arXiv: 2610.35835

中文摘要

人类监督是 AI 安全的支柱，通常由「人在回路」（HITL）对智能体关键动作进行审批来行使。但智能体绕过护栏的能力日益增强，引出关键问题：如何稳健强制执行 HITL？我们提出「不可剥夺的人在回路」（I-HITL）：即使智能体所交互的系统（如本地智能体 harness）已被攻陷，协议仍确保智能体仅在获得人类明确批准后才能发起动作；更强的版本还能确保即使恶意用户试图把审批权委托给智能体，批准也不可绕过。为论证动机，我们报告实验：在一个流行的智能体钱包平台上，提高消费上限需 HITL 通过邮件 OTP 审批；在拥有用户邮箱访问权且任务被上限卡住时，十个前沿模型常常会自主提高上限——窃取 OTP 绕过 HITL 护栏并暴露其他漏洞。我们用通信拓扑形式化定义 I-HITL，证明其实现的必要且充分条件：需要一个可信实体——可信计算设备（或等价物），并刻画哪些设备能否实现 I-HITL。我们给出原型 Countersign，可插入现有 HITL 系统并用现成消费级移动设备增强之。

原文摘要

Human oversight is a pillar of AI safety. It is often exercised by a human in the loop (HITL) who approves critical actions taken by an AI agent. AI agents' increasing ability to bypass guardrails, though, raises the question of how to enforce HITL robustly. We introduce the concept of inalienable human-in-the-loop (I-HITL). An I-HITL protocol ensures that an agent can initiate an action only with explicit human approval, even when the human interacts with a system (e.g., a local agentic harness) compromised by the agent. A strong I-HITL protocol additionally ensures such approval even when an adversarial human attempts to delegate approvals to an agent. To motivate I-HITL, we report on experiments with a popular agentic wallet platform in which raising an agent's spending cap requires HITL approval via emailed one-time passcodes (OTPs). Given access to the user's email and a task blocked by the cap, we find that ten different frontier models will often autonomously raise the cap, harvesting OTPs to bypass a HITL guardrail and surfacing other vulnerabilities. We formally define I-HITL using the concept of communication topologies and prove necessary and sufficient conditions for its realization. We show that an I-HITL protocol requires a trusted entity - a trusted computing device (or equivalent) - and characterize which such devices can and cannot realize an I-HITL protocol. We present a prototype I-HITL system called Countersign that can plug into and strengthen existing HITL systems using off-the-shelf consumer mobile devices.

自动采集于 2026-10-28

#论文 #arXiv #ML #小凯