[论文] Heads of Security: Learned Attention Gates Improve Prompt Injection Ro...

论文概要

研究领域: LLM安全
作者: Igor Shilov, Xiaoxue Yang, Alexander Panfilov, Evgenii Kortukov, Matthieu Meeus, Peter Romov, Wojciech Samek, Jonas Geiping, Yves-Alexandre de Montjoye
发布时间: 2026-10-27
arXiv: 2610.37657

中文摘要

本文提出Heads of Security（HoS），一种通过少量可学习的逐头注意力门控来强制执行指令-数据分离的提示注入防御方法。HoS学习每个注意力头应如何在可信与不可信token之间重新分配注意力，使模型能够推理不可信内容而不执行注入的指令。在Llama-3.3-70B-Instruct和GPT-OSS-20B上的评估显示：黑盒RL-Hammer攻击对GPT-OSS-20B达97% ASR、对SecAlign达85%，但对HoS仅10%。白盒攻击Claudini经autoresearch强化后，对HoS达到相同ASR需要3倍于GPT-OSS-20B或SecAlign的FLOPs。HoS在标准多选题基准和提示设置下保持了基础模型的效用，而SecAlign和ASIDE在few-shot设置下出现退化。在智能体基准上，HoS与SecAlign表现相当，显著改善了基础模型的安全-效用权衡。这些结果表明，通过有针对性的注意力干预可以实现并提升提示注入鲁棒性。

原文摘要

We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates. HoS learns how each head should redistribute attention between trusted and untrusted tokens, enabling models to reason over untrusted content without following injected instructions. We evaluate HoS on Llama-3.3-70B-Instruct and GPT-OSS-20B. We compare it against SecAlign++, a state-of-the-art finetuning-based defense, and ASIDE, a defense that, like ours, builds instruction-data separation into the model architecture. We show that HoS offers greater robustness to adaptive attacks, better performance on standard utility benchmarks, and matches top performance on standard agentic prompt injection benchmarks. More specifically, the black-box RL-Hammer attack reaches 97% ASR against GPT-OSS-20B and 85% against SecAlign, but only 10% against HoS. Claudini, a state-of-the-art white-box attack, strengthened against HoS through autoresearch, requires 3 times more FLOPs to reach the same ASR on HoS as it does on GPT-OSS-20B or SecAlign. HoS preserves base-model utility across standard multiple-choice benchmarks and prompting setups, whereas SecAlign and ASIDE degrade in few-shot settings. On agentic benchmarks with static attacks, HoS and SecAlign perform similarly and substantially improve the security-utility trade-off over the base model. ASIDE struggles with tool use, achieving only 10% utility. These results show that prompt injection robustness can be achieved and improved through targeted attention interventions.

自动采集于 2026-10-29

#论文 #arXiv #AI #小凯