RoleFlow: Self-Evolving Multi-Role Agentic Collaboration via Single-Edit Counterfactual Policy Optimization

Forum topic · 小凯 · 2026-10-24

Summary

RoleFlow is a self-evolving multi-role agentic system for LLM-based multi-agent collaboration, proposed to overcome the limits of predefined collaboration structures and the coarse credit assignment of trajectory-level rewards. It refines collaboration structures through feedback-driven atomic edits and on-demand role synthesis. To make structural adaptation learnable, the authors introduce Single-Edit Counterfactual Policy Optimization (SECPO), which evaluates each edit against its no-edit counterpart at the same chain state and assigns targeted credit to edit-specific tokens. Across seven benchmarks spanning four domains, RoleFlow achieves the best average performance, surpassing the strongest competitor by 4.7 points. It leads all three multi-hop QA benchmarks by 3.2-7.5 F1 points and also excels at mathematical reasoning, web interaction, and code generation. Further analyses demonstrate cross-benchmark transferability, training-data efficiency, and cross-executor generalization. The paper (arXiv:2610.30429) is authored by Yi Ge, Pan Wang, Xuehao Cui, Zian Zeng, Xu Hu, Heng Huang, and others.

Paper Overview

Field: Machine Learning
Authors: Yi Ge, Pan Wang, Xuehao Cui, Zian Zeng, Xu Hu, Heng Huang et al.
Published: 2026-10-24
arXiv: 2610.30429

Abstract

Large language model (LLM) based multi-agent systems often rely on predefined collaboration structures, limiting their adaptability to evolving task requirements. Although recent approaches enable dynamic orchestration, the trajectory-level rewards they apply assign one outcome to each decision, often struggling to identify the contributions of individual structural changes.

The authors propose RoleFlow, a self-evolving multi-role agentic system that refines collaboration structures through feedback-driven atomic edits and on-demand role synthesis. To make structural adaptation learnable, they introduce Single-Edit Counterfactual Policy Optimization (SECPO), which evaluates each edit against its no-edit counterpart at the same chain state and assigns targeted credit to edit-specific tokens.

Key Results

Across seven benchmarks spanning four domains, RoleFlow achieves the optimal average performance, surpassing the strongest competitor by 4.7 points.
It leads on all three multi-hop QA benchmarks by 3.2–7.5 F1 points.
It also shows supremacy on mathematical reasoning, web interaction, and code generation.
Further analyses demonstrate cross-benchmark transferability, training-data efficiency, and cross-executor generalization.

---

*Auto-collected on 2026-10-24*