[论文] KuaiRP Series Role-playing Models Technical Report

论文概要

研究领域: cs.AI, cs.CL 作者: Yipeng Wang, Ziwei Zhang, Jiahui Zhang, Qi Gan, Kai Sheng 发布时间: 2026-09-13 arXiv: 2609.11127

中文摘要

本文介绍 KuaiRP 系列角色扮演模型的完整技术方案。我们旨在为专用角色扮演模型实现四个核心目标：简化提示工程、高度稳定的输出质量、内置领域世界知识和小参数大小的高效部署。然而，有效注入深度领域知识往往导致模型一般智能体能力的严重灾难性遗忘。为克服这种权衡，我们提出多阶段训练管道。首先，我们设计标准化角色模板并基于用户行为模拟和反向配置文件过滤构建 SFT 数据管道。接下来，我们在强化学习（RL）阶段利用基于规则的复合奖励函数消除长度扩展和重复生成等常见退化现象。最后，为恢复 SFT 和 RL 期间受损的一般能力，我们提出使用配备累积发散衰减（CDD）的两阶段在线策略蒸馏（OPD）的新型自蒸馏范式。通过使用领域适应模型作为教师、原始基础模型作为学生，我们有效平衡深度领域知识注入与一般智能体能力的保留。

原文摘要

This paper introduces the complete technical solution for the KuaiRP series of role-playing models. We aim to achieve four core objectives for a dedicated role-playing model: simplified prompt engineering, highly stable output quality, built-in domain world knowledge, and high-efficiency deployment with a small parameter size. However, effectively injecting deep domain knowledge often leads to a severe catastrophic forgetting of the model's general agent capabilities. To overcome this trade-off, we propose a multi-stage training pipeline. First, we design a standardized character template and construct an SFT data pipeline based on user行为 simulation and reverse profile filtering. Next, we utilize a rule-based composite reward function during the Reinforcement Learning (RL) phase to eliminate common degradation phenomena like length expansion and repetitive generation. Finally, to recover the general capabilities compromised during SFT and RL, we propose a novel self-distillation paradigm using Two-stage On-Policy Distillation (OPD) equipped with Cumulative-Divergence Decay (CDD). By using the domain-adapted model as the teacher and the original base model as the student, we effectively balance deep domain knowledge注入 with the preservation of general agent capabilities. Experimental results demonstrate that the KuaiRP models not only match the current state-of-the-art proprietary models in role-playing fidelity within our target domains, but also successfully recover general agent capabilities, maintaining extremely low deployment costs.

---

*自动采集于 2026-09-13*

#论文 #arXiv #AI #小凯