[论文] Adversarial Closed-Loop Curriculum for Evolving Role-Playing Agents

论文概要

研究领域: ML 作者: Zheng Zhang, Liu Liu, Qi Chai 发布时间: 2026-09-25 arXiv: 2609.21994

中文摘要

基于大语言模型的角色扮演智能体已广泛应用于个性化助手和社交模拟等领域。近期的RL方法通常在开始学习前收集的固定场景池上训练，造成分布瓶颈：随着智能体进步，其表现不佳的场景在变化，而训练分布保持静态。我们提出AdvRole，一个对抗性上下文重写框架，将角色扮演RL转变为闭环课程学习。AdvRole在学习角色扮演的Actor和将角色设定与对话上下文编辑为针对性困难场景的Rewriter之间交替。Rewriter使用性能差距奖励训练，偏好能降低当前Actor得分的重写。场景池随Actor一起演化，持续针对角色-上下文空间中尚未掌握的区域。在三个涵盖英文和中文的角色扮演基准以及一个新发布的多语言基准上，AdvRole始终优于基线。

原文摘要

Role-playing agents based on large language models have been widely applied in personalized assistance and social simulation. Recent RL methods typically train on a fixed scenario pool collected before learning begins, creating a distributional bottleneck. We propose AdvRole, an adversarial context rewriting framework that turns role-playing RL into a closed-loop curriculum, alternating between an Actor that learns to role-play and a Rewriter that edits contexts into actor-specific hard scenarios.

#论文 #arXiv #ML #小凯