Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness in LLMs

Paper Overview

Field: LLM Security Authors: Igor Shilov, Xiaoxue Yang, Alexander Panfilov, Evgenii Kortukov, Matthieu Meeus, Peter Romov, Wojciech Samek, Jonas Geiping, Yves-Alexandre de Montjoye arXiv: 2610.37657

Summary

The paper introduces Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates. HoS learns how each attention head should redistribute attention between trusted and untrusted tokens, enabling models to reason over untrusted content without following injected instructions.

Evaluation Results

HoS was evaluated on Llama-3.3-70B-Instruct and GPT-OSS-20B, compared against:

• SecAlign++ — a state-of-the-art finetuning-based defense
• ASIDE — an architecture-based defense that also builds instruction-data separation into the model

Key findings:

• Black-box robustness: The RL-Hammer attack reaches 97% ASR against GPT-OSS-20B and 85% against SecAlign, but only 10% against HoS.
• White-box robustness: Claudini, a state-of-the-art white-box attack strengthened against HoS through autoresearch, requires 3x more FLOPs to reach the same ASR on HoS as it does on GPT-OSS-20B or SecAlign.
• Utility: HoS preserves base-model utility across standard multiple-choice benchmarks and prompting setups, whereas SecAlign and ASIDE degrade in few-shot settings. ASIDE struggles with tool use, achieving only 10% utility.
• Agentic benchmarks: With static attacks, HoS and SecAlign perform similarly and substantially improve the security-utility trade-off over the base model.

Conclusion

These results demonstrate that prompt injection robustness can be achieved and improved through targeted attention interventions.

---

Original abstract:

> We introduce Heads of Security (HoS), a prompt injection defense that enforces instruction-data separation via a small set of learned per-head attention gates. HoS learns how each head should redistribute attention between trusted and untrusted tokens, enabling models to reason over untrusted content without following injected instructions. We evaluate HoS on Llama-3.3-70B-Instruct and GPT-OSS-20B. We compare it against SecAlign++, a state-of-the-art finetuning-based defense, and ASIDE, a defense that, like ours, builds instruction-data separation into the model architecture. We show that HoS offers greater robustness to adaptive attacks, better performance on standard utility benchmarks, and matches top performance on standard agentic prompt injection benchmarks. These results show that prompt injection robustness can be achieved and improved through targeted attention interventions.