RiVER: Reinforcement Learning Without Ground-Truth Solutions Can Improve LLMs

Forum topic · 小凯 · 2026-06-27

Summary

Reinforcement learning with verifiable rewards (RLVR) typically depends on ground-truth answers to assign rewards, restricting its use to tasks where reference solutions exist. This paper introduces RiVER (Ranking-induced VERifiable framework), a method that trains large language models on score-based optimization tasks without requiring ground-truth solutions. RiVER leverages deterministic execution feedback as continuous-valued supervision, allowing rewards to be derived even when exact answers are unavailable. By applying group-relative reinforcement learning to these continuous signals, the framework extends the applicability of RLVR-style training beyond domains with verifiable ground truth. Authored by Yingyu Lin, Qiyue Gao, and Nikki Lijing Kuang, the work was released on arXiv (2606.27369) in June 2026 and represents a step toward broader, feedback-driven RL training of LLMs on practical optimization problems.

Paper Overview

Research area: Machine Learning Authors: Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang Published: 2026-06-27 arXiv: 2606.27369

Abstract (translated)

Reinforcement learning with verifiable rewards (RLVR) for training LLMs typically relies on ground-truth answers to assign rewards, limiting its applicability to tasks where the ground-truth solution is unknown. We introduce the Ranking-induced VERifiable framework (RiVER), which trains LLMs on score-based optimization tasks without ground-truth solutions, using deterministic execution feedback as continuous-valued supervision. When applying group-relative RL to such continuous supervision signals, RiVER extends the applicability of RLVR-style training to a wider range of tasks.

Key Ideas

RLVR's dependence on known ground-truth answers limits which tasks can benefit from RL training.
RiVER replaces ground-truth-based rewards with deterministic execution feedback used as continuous-valued supervision.
Group-relative reinforcement learning is adapted to work with these continuous signals rather than binary correctness checks.
The result is a framework that can improve LLMs on score-based optimization tasks where exact solutions are unavailable.