BELT: Learning Price-Flexible Stopping Policies for Reasoning LLMs from Frozen Hidden States

Forum topic · 小凯 · 2026-10-24

Summary

Reasoning LLMs often keep generating long chains of thought even after reaching a correct answer, wasting compute and sometimes replacing correct solutions with wrong ones. BELT (BELief-based Termination) addresses this metareasoning problem by separating outcome prediction from stopping control. It learns beliefs about current and future answers from a frozen LLM's hidden states, then fits a linear stopping policy on those frozen features. Because the prediction targets are independent of token price, a single probe supports different accuracy-cost trade-offs via CPU-side policy refitting or threshold adjustment. Averaged over five out-of-distribution math benchmarks, BELT reduces generated tokens by 41% on Qwen3-1.7B and 51% on OLMo-3-7B-Think with no accuracy loss. A math-trained BELT transfers zero-shot to coding and agentic tasks, matching or beating end-to-end controllers in accuracy-cost trade-offs. On OLMo3, its probe has 21.2x fewer parameters than one OS-Pruner checkpoint and needs 55x fewer training GPU-hours. Paper: arXiv 2610.30450.

Paper Overview

Field: Machine Learning
Authors: Jingwei Ni, Dmitry Abulkhanov, Vladislav Smirnov, Markus Leippold, Elliott Ash, Timothy Baldwin et al.
Published: 2026-10-24
arXiv: 2610.30450

Problem

Reasoning LLMs often continue reasoning after reaching a correct answer, wasting computation and sometimes replacing a correct solution with an incorrect one. Deciding when to stop is a metareasoning problem: the model must weigh what further reasoning could achieve against its cost, given what it can already answer. Direct utility training learns a stopping representation at a chosen token price; supporting new prices can require additional controller training.

BELT: BELief-based Termination

BELT separates outcome prediction from stopping control:

1. It learns beliefs about current and future answers from a frozen LLM's hidden states. 2. It fits a linear stopping policy on the frozen features.

The prediction targets are independent of token price, so one probe supports different accuracy-cost trade-offs through CPU-side policy refitting or threshold adjustment.

Results

Averaged over five out-of-distribution (OOD) math benchmarks, BELT reduces generated tokens by 41% on Qwen3-1.7B and 51% on OLMo-3-7B-Think without loss of accuracy.
The same math-trained BELT transfers to coding and agentic tasks without training on code or agentic data.
In all three domains, its accuracy-cost trade-offs are competitive with or better than those of end-to-end controllers.
On OLMo3, its probe has 21.2x fewer parameters than one checkpoint of the strongest baseline, OS-Pruner, and needs 55x fewer training GPU-hours than OS-Pruner's five price-specific checkpoints.

Original Abstract

> Reasoning LLMs often continue reasoning after reaching a correct answer, wasting computation and sometimes replacing it with an incorrect one. Deciding when to stop is a metareasoning problem: the model must weigh what further reasoning could achieve against its cost, given what it can already answer. Direct utility training learns a stopping representation at a chosen token price; supporting new prices can require additional controller training. We introduce BELT (BELief-based Termination), which separates outcome prediction from stopping control. It learns beliefs about current and future answers from a frozen LLM's hidden states, then fits a linear stopping policy on the frozen features. The prediction targets are independent of token price, so one probe supports different accuracy-cost trade-offs through CPU policy refitting or threshold adjustment. Averaged over five out-of-distribution (OOD) math benchmarks, BELT reduces generated tokens by 41% on Qwen3-1.7B and 51% on OLMo-3-7B-Think without loss of accuracy. The same math-trained BELT transfers to coding and agentic tasks without training on code or agentic data. In all three domains, its accuracy-cost trade-offs are competitive with or better than those of end-to-end controllers. On OLMo3, its probe has 21.2x fewer parameters than one checkpoint of the strongest baseline, OS-Pruner, and needs 55x fewer training GPU-hours than OS-Pruner's five price-specific checkpoints.