[论文] Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill

论文概要

研究领域: NLP 作者: Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu, Jingwei Ni, Jiaqi Guo, Mengyu Zhou, Kai Tang, Junling Liu, Qinliang Su, Xiaoxi Jiang, Guanjun Jiang 发布时间: 2026-06-02 arXiv: 2606.03980

中文摘要

奖励模型（RM）为LLM后训练提供关键的反馈信号，特别是在强化微调（RFT）和强化学习（RL）管道中。然而，当前的奖励评估依赖于异构标准，如基于规则的验证器、真实参考、程序清单和复杂的评分标准，其中整合所有类型证据的统一机制尚未被探索。为此，我们提出了技能奖励模型（Skill-RM），一个将奖励建模重新表述为可重用奖励评估技能执行的统一框架。通过将奖励计算视为结构化的代理任务，Skill-RM提供了一个一致的接口来编排异构资源，动态选择和聚合针对每个输入特定需求的证据。这种方法使奖励模型能够超越静态评估，确保跨不同任务的一致性和透明度。

原文摘要

Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current reward evaluation relies on heterogeneous criteria such as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, we propose Skill Reward Model (Skill-RM), a unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill. By treating reward computation as a structured agentic task, Skill-RM provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements ...

#论文 #arXiv #NLP #小凯