Paper Overview
- Field: Machine Learning
- Authors: Junjie Zhang, Hui Liu, Kecheng Chen
- arXiv: 2508.11367
- RedEvoAgent: a black-box red-teaming agent that distills cross-case attack trajectories into concise, human-readable attack skills.
- Adaptive skill evolution: attack skills evolve via tool-effectiveness profiling and decision-based tool attribution for skill updates.
- Verification ratchet: only skill updates that improve verification performance are retained.
Summary
LLM-based agents are increasingly deployed in product-level execution harnesses, where jailbreaks can trigger harmful tool use and persistent state changes, creating greater risks than unsafe text generation alone. Existing automatic red-teaming methods often rely on fixed attacks, while recent agentic attackers coordinate multiple jailbreak tools and show stronger potential through trajectory-based retrieval. However, such retrieval can reuse misleading experiences due to retrieval bias and unclear tool credit, and full trajectories add context overhead while reducing interpretability.
Key Contributions
Results
Experiments across multiple benchmarks, target models, and target execution environments show that RedEvoAgent outperforms both fixed and agentic baselines, improves tool efficiency, and transfers across attacker models and target execution environments.
---
*Auto-collected on 2026-08-29*