English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original
智柴网 中文原文

Route-and-Generate (Rouge): KL-Regularized Router for Adaptive LLM Reasoning Effort

Forum topic · 小凯 · 2026-10-24

Summary

Large language models often apply complex reasoning indiscriminately, overthinking simple tasks and underthinking hard ones. Route-and-Generate (Rouge), presented in arXiv paper 2610.30351 by Nguyen et al., is a hierarchical training framework that lets an LLM adaptively decide when and how much reasoning each input requires. The framework decouples reasoning effort allocation from answer generation: a routing policy allocates a fixed sampling budget across multiple reasoning formats while the base model is optimized with GRPO using per-sample rewards. Efficiency is promoted through a rank-based reward favoring shorter correct solutions, and the routing policy is updated via a closed-form KL-regularized policy improvement step distilled into a neural router. This design yields concise answers on easy problems while reserving long chain-of-thought reasoning for harder inputs, delivering improved accuracy-efficiency trade-offs across multiple reasoning benchmarks.

Paper Overview

Field: Machine Learning Authors: Tien-Phat Nguyen, Truong Nguyen, Thanh-Thien Le, Hoang Phan, Thien Huu Nguyen, Linh Ngo Van et al. Published: 2026-10-24 arXiv: 2610.30351

Summary

Large language models often apply complex reasoning indiscriminately, leading to overthinking on simple tasks and underthinking on harder ones. This paper presents Route-and-Generate (Rouge), a hierarchical training framework that enables LLMs to adaptively decide *when* and *how much* reasoning is required for each input.

Method Highlights

  • Decoupled design: Reasoning effort allocation is separated from answer generation.
  • Routing policy: Allocates a fixed sampling budget across multiple reasoning formats.
  • Base model optimization: Trained with GRPO using per-sample rewards.
  • Efficiency reward: A rank-based reward favors shorter correct solutions, encouraging concise answers on easy problems while reserving longer reasoning for harder ones.
  • Router update: The routing policy is improved via a closed-form KL-regularized policy improvement step, distilled into a neural router.

Results

The framework achieves improved accuracy-efficiency trade-offs across multiple reasoning benchmarks.

Original Abstract

> Large language models often apply complex reasoning indiscriminately, leading to overthinking on simple tasks and underthinking on harder ones. We present Route-and-Generate (Rouge), a hierarchical training framework that enables LLMs to adaptively decide when and how much reasoning is required for each input. Our approach decouples reasoning effort allocation from answer generation by introducing a routing policy that allocates a fixed sampling budget across multiple reasoning formats, while the base model is optimized with GRPO using per-sample rewards. To promote efficient reasoning, we employ a rank-based reward that favors shorter correct solutions, and update the routing policy via a closed-form KL-regularized policy improvement step distilled into a neural router. This design encourages concise answers on easy problems while reserving longer reasoning for harder ones, achieving improved accuracy-efficiency trade-offs across multiple reasoning benchmarks.

--- *Auto-collected on 2026-10-24*

Tags

#llm#reasoning#reinforcement-learning#grpo#adaptive-computation#efficiency#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636386