AdvSim2Real: Co-Evolutionary Training of Web Agents Against Adaptive Prompt Injection

Forum topic · 小凯 · 2026-10-08

Summary

AdvSim2Real (arXiv:2610.08773) is a training framework that hardens web agents against prompt injection attacks. Web agents read and act on third-party pages, so planted instructions can hijack them from the user's goal, yet the agent cannot ignore pages since they contain required values and controls. Existing defenses fine-tune agents on injections fixed before training, which adaptive attackers bypass, while adversarial training keeps tasks fixed so they stop teaching once solved. AdvSim2Real co-evolves a task curriculum, an injection adversary, and the agent inside a frozen web world model: the curriculum is rewarded for tasks the agent solves roughly half the time, and the adversary only for success flips that turn task success into failure. Training a 4B agent in the simulator yields both higher capability and robustness—completion rates rise with and without attacks, the agent resists frontier-model adversaries it never trained against, and capability gains transfer to real browsers. Across 150 web tasks, AdvSim2Real improves completion rate against an unseen adversary by 33.6% relative to the base agent.

论文概要

研究领域: NLP 作者: Sarim Hashmi, Mukul Ranjan, Kshitij Mishra, Mikhail Kuznetsov, Praneeth Vepakomma, Nils Lukas arXiv: 2610.08773

Why Existing Defenses Fall Short

Web agents complete user requests by reading and acting on pages that third parties write, so an instruction planted on a page can redirect the agent away from the user's goal. The agent cannot simply ignore the page, because the page also holds the values and controls the task requires.

Current defenses fine-tune agents on injections fixed before training—attackers that adapt to the trained model bypass them.
Adversarial training lets the attacker adapt, but keeps tasks fixed, so a task stops teaching once the agent solves it.

The AdvSim2Real Approach

AdvSim2Real co-evolves three components inside a frozen web world model:

1. Task curriculum — rewarded for tasks the agent solves about half of the time, keeping difficulty at the frontier of the agent's ability. 2. Injection adversary — rewarded only for a *success flip*, i.e., an injection that turns task success into failure. 3. The agent — trained against this escalating pressure.

Results

Simulator training makes a 4B agent both more capable and more robust.
Completion rates rise with and without attacks.
The agent resists frontier-model adversaries it was never trained against.
Capability gains transfer to real browsers.
Across 150 web tasks, AdvSim2Real improves completion rate against an unseen adversary by 33.6% relative to the base agent.

References

Paper: arXiv:2610.08773