Skill-RM: A Unified Framework Unifying Heterogeneous Reward Evaluation Criteria via Agentic Skills

Forum topic · 小凯 · 2026-06-04

Summary

Skill-RM is a research paper (arXiv:2606.03980) that proposes a unified framework for reward modeling in LLM post-training. Reward models supply crucial feedback signals in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines, but current reward evaluation relies on heterogeneous criteria—including rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics—and no unified mechanism exists to integrate all types of evidence. Skill-RM reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill, treating reward computation as a structured agentic task. This provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input. The approach enables reward models to move beyond static evaluation, ensuring consistency and transparency across diverse tasks. The paper is authored by Tao Chen, Gangwei Jiang, Pengyu Cheng, and colleagues, published June 2, 2026, in the NLP domain.

Paper Overview

Research area: NLP Authors: Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu, Jingwei Ni, Jiaqi Guo, Mengyu Zhou, Kai Tang, Junling Liu, Qinliang Su, Xiaoxi Jiang, Guanjun Jiang Published: 2026-06-02 arXiv: 2606.03980

Abstract

Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current reward evaluation relies on heterogeneous criteria such as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, the authors propose Skill Reward Model (Skill-RM), a unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill.

Key Ideas

Problem: Reward evaluation criteria in LLM training pipelines are heterogeneous (rules, ground-truth references, checklists, rubrics), and no unified mechanism exists to integrate all evidence types.
Approach: Skill-RM treats reward computation as a structured agentic task, reformulating reward modeling as the execution of a reusable Reward-Evaluation Skill.
Mechanism: The framework provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input.
Benefits: Moves beyond static evaluation, ensuring consistency and transparency of reward assessment across diverse tasks.