[论文] RoleFlow: Self-Evolving Multi-Role Agentic Collaboration via Structura...

论文概要

研究领域: ML 作者: Yi Ge, Pan Wang, Xuehao Cui, Zian Zeng, Xu Hu, Heng Huang et al. 发布时间: 2026-10-24 arXiv: 2610.30429

中文摘要

基于 LLM 的多智能体系统常依赖预定义的协作结构，难以适应不断演化的任务需求。近期方法虽支持动态编排，但它们施加的轨迹级奖励把每个结果指派给一连串决策，难以识别单个结构变更的贡献。本文提出 RoleFlow：通过反馈驱动的原子编辑与按需角色合成实现自演化的多角色智能体协作。为使结构适应可学习，引入单编辑反事实策略优化（SECPO）——在同一链状态下对每个编辑与其「不编辑」对应物对比评估，把定向信用分配给编辑特定的 token。跨四个领域七个基准，RoleFlow 取得最优平均性能，超出最强竞争者 4.7 分；在三个多跳 QA 基准上以 3.2–7.5 个 F1 领先，在数学推理、网页交互与代码生成上同样占优。进一步分析显示跨基准可迁移性、训练数据效率与跨执行器泛化能力。

原文摘要

Large language model (LLM) based multi-agent systems often rely on predefined collaboration structures, limiting their adaptability to evolving task requirements. Although recent approaches enable dynamic orchestration, trajectory-level rewards they apply assign one outcome to each decision, ofen struggling to identify the contributions of individual structural changes. We propose RoleFlow, a self-evolving multi-role agentic system that refines collaboration structures through feedback-driven atomic edits and on-demand role synthesis. To make structural adaptation learnable, we introduce Single-Edit Counterfactual Policy Optimization (SECPO), which evaluates each edit against its no-edit counterpart at the same chain state and assigns targeted credit to edit-specific tokens. Across seven benchmarks spanning four domains, RoleFlow achieves the optimal average performance, surpassing the strongest competitor by 4.7 points. It leads on all three multi-hop QA benchmarks by 3.2-7.5 F1 points and also shows supremacy on mathematical reasoning, web interaction, and code generation. Further analyses demonstrate cross-benchmark transferability, training-data efficiency, and cross-executor generalization.

---

*自动采集于 2026-10-24*

#论文 #arXiv #ML #小凯