RiVER: Reinforcement Learning Without Ground-Truth Solutions Improves LLMs

Forum topic · 小凯 · 2026-06-27

Summary

This forum post summarizes an arXiv paper (2606.27369) introducing RiVER (Ranking-induced VERifiable framework), a reinforcement learning method for training LLMs on score-based optimization tasks without ground-truth solutions. Standard RLVR (reinforcement learning with verifiable rewards) depends on known answers to assign rewards, restricting its use to tasks with verifiable ground truth. RiVER instead uses deterministic execution feedback as continuous-valued supervision, enabling training in settings where the correct solution is unknown. Applying group-relative RL to continuous rewards, the framework broadens the applicability of verifiable-reward training to execution-checkable but unsolved tasks. Authors: Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang. Posted on zhichai.net, collected 2026-06-27.

Paper Overview

Field: Machine Learning
Authors: Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang
arXiv: 2606.27369
Posted: 2026-06-27 (auto-collected)

What the Paper Proposes

The authors introduce RiVER (Ranking-Induced VERifiable framework), which trains LLMs on score-based optimization tasks without ground-truth solutions, using deterministic execution feedback as continuous-valued supervision.

Key Idea

Instead of comparing against a known answer, RiVER executes candidate outputs and uses the resulting deterministic feedback as a continuous reward signal.
Group-relative RL is applied to these continuous rewards to optimize the model.
This extends RLVR-style training to domains that are execution-checkable but lack a ground-truth solution.

Significance

RiVER broadens the range of tasks to which verifiable-reward reinforcement learning can be applied, allowing LLM improvement in settings where answers are unknown but outcomes can be deterministically evaluated by execution.

Paper link: https://arxiv.org/abs/2606.27369