Consilience for Verifier-Free Test-Time Scaling: Rethinking Confidence in LLM Reasoning

Forum topic · 小凯 · 2026-08-12

Summary

This paper (arXiv 2508.05137) by Lecheng Kong, Like Hui, and Haitao Mao addresses verifier-free test-time scaling (VF-TTS) for enhancing LLM reasoning. Test-time scaling typically relies on external verifiers such as compilers, test cases, or trained value functions, but such verifiers are unavailable in many real-world applications. Among VF-TTS approaches, confidence-based methods that rank rollouts solely by confidence are attractive due to near-zero evaluation overhead and minimal reliance on internal model states. The authors identify a critical limitation: these methods catastrophically fail on complex tasks, where uniformly high confidence often signals failed exploration and confident wrong answers. Their key insight is that robust epistemic search requires a specific confidence trajectory pattern—exploratory branching (low initial confidence) that converges to high final confidence. They propose consilience, a selection framework that explicitly evaluates temporal asymmetry in confidence during reasoning using a combined metric that penalizes high initial confidence while requiring final certainty. Experiments on graduate-level math problems and free-form code generation show consilience outperforms existing baselines, validating this new perspective on completion confidence.

Paper Overview

Field: NLP
Authors: Lecheng Kong, Like Hui, Haitao Mao
Published: 2026-08-12
arXiv: 2508.05137

Abstract

Test-time scaling often uses an external verifier, such as compilers and test cases in coding or trained value functions in robotics applications, to obtain high-quality rollouts. Verifier-free test-time scaling (VF-TTS) is gaining extensive attention as a mechanism to enhance Large Language Model (LLM) reasoning, primarily because we do not have access to such high-quality verifiers in many real-world applications.

Among existing VF-TTS methods, confidence-based VF-TTS methods, which compute and rank rollouts solely by confidence, are particularly promising. Such methods introduce near-zero overhead for sample evaluation and need minimal access to internal model states, making the methods highly flexible across models and tasks.

In this paper, the authors demonstrate a critical limitation of existing confidence-based VF-TTS methods: they can catastrophically collapse on complex tasks. An interesting observed phenomenon is that uniformly high confidence usually indicates failed exploration, tending toward confidently wrong answers.

Key Insight

Robust epistemic search requires a specific confidence trajectory pattern: methods should perform exploratory branching at the start (manifesting as low initial confidence) and converge to high final-confidence solutions.

Method: Consilience

To realize this insight, the authors introduce consilience, a novel selection framework that explicitly evaluates temporal asymmetry of confidence during reasoning. It is implemented via a combined metric that:

Actively penalizes high initial confidence
Strictly requires final certainty

Results

Extensive experiments on graduate-level math problems and free-form code generation show that consilience effectively outperforms existing baselines, validating this novel perspective on completion confidence.

---

*Auto-collected on 2026-08-12.*