[论文] Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolvi...

论文概要

研究领域: NLP
作者: Siyuan Huang, Pengyu Cheng, Haotian Liu, Tao Chen, Yihao Liu, Jingwei Ni, Shijie Zhou, Ziyi Yang, Gangwei Jiang, Mengyu Zhou, Yu Cheng, Xiaoxi Jiang, Guanjun Jiang
发布时间: 2026-07-24
arXiv: 2607.22529

中文摘要

LLM训练正从手动设计和标注转向交互驱动的自我进化。然而，现有的自进化方法面临任务多样性与验证可靠性之间的根本性困境：基于环境的方法获得精确反馈但将学习限制在狭窄领域，而开放式自生成拓宽了任务空间但缺乏可靠验证，允许误导性奖励污染训练循环。本文将智能体技能视为调和这种张力的有力中间地带：每个技能确保在特定场景中的深度、可验证执行，而跨技能的动态路由保持开放式任务多样性。利用这一洞察，本文引入技能自博弈(Skill-SP)，一个包含提议者、求解器和动态技能控制器的协同进化框架。通过强化学习循环 orchestrated，这些组件在持续的自博弈循环中协同进化：提议者基于动态采样的技能生成挑战性任务；求解器探索候选解决方案以推动其能力边界；技能控制器收集执行反馈以更新和扩展技能库。这种交互式协同进化有效弥合了结构化验证与开放式探索之间的差距。在工具使用和推理基准测试上的实证评估表明，Skill-SP作为一个稳健进化引擎，持续提升胜任骨干模型的性能上限，同时催化初始未对齐模型的显著逆转。

原文摘要

LLM training is shifting from manual design and annotation to interaction-driven self-evolution. However, existing self-evolutionary methods face a fundamental dilemma between task diversity and verification reliability: environment-bound methods obtain precise feedback but confine learning to narrow domains, while open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop. We identify agent skills as a powerful middle ground to reconcile this tension: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills maintains open-ended task variety. Leveraging this insight, we introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a s...

自动采集于 2026-07-28

#论文 #arXiv #NLP #小凯