RACES: Recursive Composition of Verifiable Environments for Scaling RL-Based LLM Reasoning

Forum topic · 小凯 · 2026-06-12

Summary

RACES (Recursive Automated Composition for Environment Scaling) is a framework that treats verifiable reinforcement learning environments as composable building blocks for training LLM reasoning. Instead of manually constructing environments one by one, RACES automatically fuses environments whenever one environment's output type matches another's input type, enabling recursive composition. The framework is implemented with 300 independent environments and defines composition operators (SEQUENTIAL, PARALLEL, SORT, SELECT) that induce diverse reasoning patterns. Experiments show RL training on composed environments consistently improves reasoning generalization: DeepSeek-R1-Distill-Qwen-14B improves by 3.1 points on average (48.2 to 51.3) across six unseen benchmarks, and Qwen3-14B rises from 58.8 to 61.1. Notably, RACES with only 50 base environments matches the performance of training on 300 independent environments, demonstrating substantial efficiency gains in environment utilization. Paper: arXiv 2606.12373.

Overview

Research area: NLP Authors: Hao Xiang, Qiaoyu Tang, Le Yu, Yaojie Lu, Xianpei Han, Ben He, Le Sun, Bowen Yu, Peng Wang, Hongyu Lin, Dayiheng Liu Published: 2026-06-10 arXiv: 2606.12373

Abstract

Reinforcement Learning (RL) with verifiable environments has emerged as a powerful approach for enhancing the reasoning capabilities of Large Language Models (LLMs). While prior research demonstrates that scaling environment quantity improves RL performance, existing manual or individual construction methods suffer from linear scaling limits, thereby hindering scalable reasoning generalization.

This paper introduces RACES (Recursive Automated Composition for Environment Scaling), a framework that conceptualizes verifiable environments as composable building blocks that can be recursively assembled. The key insight is that when the codomain (output type) of one environment matches the domain (input type) of another, they can be automatically fused into a new verifiable environment, enabling recursive composition.

Method

RACES is implemented with 300 independent environments.
A set of composition operators — SEQUENTIAL, PARALLEL, SORT, and SELECT — is defined to induce diverse reasoning patterns.
Environments compose automatically whenever one environment's codomain (output type) matches another's domain (input type).

Results

Training RL on composed environments consistently improves reasoning generalization.
DeepSeek-R1-Distill-Qwen-14B improves by 3.1 points on average (48.2 → 51.3) across six benchmarks unseen during training environment construction.
Qwen3-14B improves from 58.8 to 61.1.
RACES with only 50 base environments matches the performance of training on 300 independent environments, demonstrating significant efficiency in environment utilization.

---

*Auto-collected on 2026-06-12.*