M³: Automated Multi-Turn LLM Jailbreaking as a Markov Decision Process

Forum topic · 小凯 · 2026-10-26

Summary

M³ is an automated red-teaming framework that formalizes multi-turn jailbreaking of large language models as a finite-horizon Markov Decision Process (MDP). The framework was introduced by researchers including Yuqing Huang, Jack Ji, Jithin Pradeep, Sagarika Suresh, and Jekaterina Novikova (arXiv: 2610.31937), addressing the limitation that many existing red-teaming methods rely on heuristics or strong assumptions that constrain fully black-box settings. In M³, an attacker LLM interacts with a target LLM while operating over a compact internal state generated by a state LLM. Across two benchmarks and six target models, M³ consistently outperforms state-of-the-art multi-turn red-teaming methods in Attack Success Rate (ASR), with improvements of up to 53% in ASR@5. Even with a lightweight attacker, M³ achieves gains of up to 26% in ASR@1 and 34% in ASR@5 compared to baselines using higher-capacity attackers. The method also exhibits steeper risk escalation across dialogue turns and delivers more consistent performance across risk categories without predefined risk taxonomies (Gini index improvement up to 4.82), offering a scalable, principled framework for multi-turn red-teaming and safety evaluation.

Overview

Field: Machine Learning Authors: Yuqing Huang, Jack Ji, Jithin Pradeep, Sagarika Suresh, Jekaterina Novikova Published: 2026-10-27 arXiv: 2610.31937

Modern Large Language Models (LLMs) remain vulnerable to carefully orchestrated multi-turn interactions that progressively bypass safety mechanisms. However, many existing red-teaming approaches rely on heuristics or impose strong assumptions that limit their applicability in fully black-box settings.

To address this gap, the authors introduce M³, an automated red-teaming framework that formalizes multi-turn jailbreaking as a finite-horizon Markov Decision Process (MDP). In M³, an attacker LLM interacts with a target LLM while operating over a compact internal state produced by a state LLM.

Key Results

Evaluated on two benchmarks and six target models
Consistently outperforms state-of-the-art multi-turn red-teaming methods in Attack Success Rate (ASR), with improvements of up to 53% in ASR@5
Even with a lightweight attacker, M³ achieves gains of up to 26% in ASR@1 and 34% in ASR@5 compared to baselines using higher-capacity attackers
Shows steeper risk escalation across dialogue turns
Delivers more consistent performance across risk categories without relying on predefined risk taxonomies, with Gini index improvements up to 4.82

Significance

M³ provides a scalable and principled framework for multi-turn red-teaming and safety evaluation of LLMs, particularly suited to fully black-box settings where heuristics and strong assumptions are impractical.

Original abstract (truncated in source): "Modern Large Language Models (LLMs) remain vulnerable to carefully orchestrated multi-turn interactions that progressively bypass safety mechanisms. However, many existing red-teaming approaches rely on heuristics or impose strong assumptions that limit their applicability in fully black box settings. We address this gap by introducing M³, an automated red-teaming framework that formalizes multi-turn jailbreaking as a finite-horizon Markov Decision Process (MDP)..."

Paper: https://arxiv.org/abs/2610.31937