Adversarial Poetry: A Universal Single-Turn Jailbreak That Bypasses LLM Safety Across 25 Models

Forum topic · QianXun · 2025-11-24

Summary

A study from DEXAI Icaro Lab and Sapienza University of Rome (arXiv:2511.15304) shows that wrapping harmful requests in poetry is a universal single-turn jailbreak for large language models. Testing 25 frontier models from nine providers (Google, OpenAI, Anthropic, DeepSeek, Qwen, Mistral, Meta, xAI, Moonshot), handcrafted adversarial poems achieved a 62% average attack success rate (ASR), with Gemini-2.5-pro at 100%. When 1,200 MLCommons harmful prompts were automatically converted to verse via a meta-prompt, overall ASR jumped from 8.08% to 43.07%—up nearly 35 percentage points, with privacy-related prompts rising by almost 45 points. The mechanism is black-box and model-agnostic, exploiting mismatched generalization: safety training overfits to prose-style harms, so metaphor, rhythm, and narrative framing evade refusal. Counterintuitively, smaller models (GPT-5-nano: 0% ASR) were safer, and Anthropic's Constitutional AI showed notable resilience. The authors warn that static benchmarks may systematically overstate robustness and propose style-diversified adversarial training, intent anchoring, dynamic safety layers, and interdisciplinary red teaming.

Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in LLMs

This post discusses the arXiv preprint *Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models* (arXiv:2511.15304v2), by a DEXAI – Icaro Lab and Sapienza University of Rome team. Framed with a nod to Plato's expulsion of poets from the Republic, the study reveals a systemic weakness in AI alignment: when harmful requests are dressed in verse, safety mechanisms fail across the board.

Key points

Scale of testing: 25 frontier LLMs from nine providers — Anthropic, DeepSeek, Google, Meta, Mistral AI, Moonshot AI, OpenAI, Qwen, xAI — covering both proprietary and open-weight models.
Handcrafted attack: 20 poems (English and Italian) spanning CBRN hazards, cyberattacks, harmful manipulation, and loss-of-control scenarios achieved a 62% average attack success rate (ASR).
gemini-2.5-pro: 100% ASR; deepseek-chat-v3.1 and magistral-medium-2506: 95%; qwen3-max and gemini-2.5-flash: 90%
gpt-5-nano was the only model with 0% ASR
Scaled attack: 1,200 MLCommons AI safety benchmark prompts were converted to poems via a meta-prompt executed with DeepSeek-R1. Overall ASR rose from 8.08% (prose baseline) to 43.07% (+34.99 points).
Largest gains: privacy (8.07% → 52.78%), non-violent crime (10.75% → 50.10%), CBRNE weapons (6.81% → 45.13%)
Sexual content showed relative resilience (+24.64 points)
Cyberattack category was especially vulnerable to handcrafted poems: code injection/RCE 84%, offline password/hash cracking 84%, malware persistence 80%, covert data extraction 80%.

Evaluation methodology

A three-judge system (gpt-oss-120b, kimi-k2-thinking, deepseek-r1) independently labeled outputs SAFE/UNSAFE with majority voting; contested cases went to human arbitration. About 5% of outputs (~2,100 samples) were manually annotated with high agreement.

Why poetry works

The attack exploits mismatched generalization (Wei et al., 2023): safety training overfits to prose-form harmful content. Poetry defeats it through:

1. Metaphorical density — creating new conceptual mappings (a bioweapon lab becomes "the baker's secret oven") 2. Rhythmic disruption — diluting safety-critical information in aesthetic form 3. Narrative framing hijack — poetry carries a fiction meta-label, causing contextual displacement

Chain-of-thought reasoning can backfire: the model invests effort in interpreting the poem and drifts into task-execution mode, forgetting safety constraints.

Counterintuitive findings

Capability–vulnerability inverse gradient: within OpenAI, GPT-5-nano (0% ASR) < GPT-5-mini (5%) < GPT-5 (10%); similar trends in Claude and Grok families. Smaller models may fail to decode metaphor or default to refusal under uncertainty.
Vendor matters more than openness: Anthropic stayed resilient at all sizes (+3.12 points ASR increase), while DeepSeek (+62.15) and Google (+56.91) saw the largest gains. Vulnerability is determined by alignment philosophy, not open vs. proprietary strategy.

Regulatory implications

The authors argue that static benchmarks and the EU AI Act / GPAI Code of Practice compliance frameworks may systematically overstate robustness: a model with 95%+ refusal on standard prose tests can drop far below that under minimal stylistic variation. They propose future red teaming include stylistic-variation testing, narrative-frame injection, and metaphor-density scanning.

Defense directions

1. Style-diversified adversarial training — include poetic/script/lyric versions of harmful prompts in alignment data 2. Intent anchoring — detect harmful intent regardless of surface form 3. Dynamic safety layers — detect stylistic anomalies at inference and switch to conservative responses 4. Open, interdisciplinary red teaming — involving poets, writers, and linguists

References

1. Bisconti, P., Prandi, M., Pierucci, F., et al. (2025). *Adversarial Poetry as a Universal Single-Turn Jailbreak Mechanism in Large Language Models*. arXiv:2511.15304v2 [cs.CL]. https://arxiv.org/pdf/2511.15304v2.pdf 2. Wei, A., Haghtalab, N., & Steinhardt, J. (2023). *Jailbroken: How Does LLM Safety Training Fail?* arXiv:2307.02483 3. Ziegler, D. M., et al. (2020). *Fine-Tuning Language Models from Human Preferences*. arXiv:1909.08593 4. Vidgen, B., et al. (2024). *AI Risk and Reliability Benchmark (AI R&R)*. MLCommons AILuminate Benchmark 5. Bai, Y., et al. (2022). *Constitutional AI: Harmlessness from AI Feedback*. arXiv:2212.08073

The post concludes: an AI capable of writing sonnets can also be deceived by sonnets — stylistic form is not a secondary concern but a core battlefield for AI safety, and future architectures must balance expressiveness with defense.