Consilience: A Verifier-Free Test-Time Scaling Framework Using Confidence Trajectory Asymmetry

Forum topic · 小凯 · 2026-08-12

Summary

This paper addresses verifier-free test-time scaling (VF-TTS) for enhancing Large Language Model reasoning without external verifiers such as compilers or trained value functions. The authors focus on confidence-based VF-TTS methods, which rank candidate rollouts solely by model confidence and require near-zero sample evaluation overhead. They identify a critical limitation: existing methods suffer catastrophic collapse on complex tasks because uniformly high confidence often signals exploration failure, leading the model to commit to confidently wrong answers. Their key insight is that robust epistemic search requires a specific confidence trajectory—exploratory branching with low initial confidence followed by convergence to high final confidence. To operationalize this, they introduce Consilience, a selection framework that explicitly evaluates temporal asymmetry in reasoning confidence by penalizing high initial confidence while demanding strong final certainty. Experiments on graduate-level mathematics and free-form code generation demonstrate that Consilience consistently outperforms existing confidence-based baselines, validating a novel perspective on completion confidence.

Paper Overview

Research Area: NLP Authors: Lecheng Kong, Like Hui, Haitao Mao Release Date: 2026-08-12 arXiv: 2508.05137

Abstract

Test-time scaling often uses an external verifier—such as compilers and test cases in coding, or trained value functions in robotics—to obtain high-quality rollouts. Verifier-free test-time scaling (VF-TTS) is gaining extensive attention as a mechanism to enhance Large Language Model (LLM) reasoning, primarily because high-quality verifiers are unavailable in many real-world applications. Among existing VF-TTS methods, confidence-based VF-TTS methods, which compute and rank rollouts solely by confidence, are particularly promising. These methods introduce near-zero overhead for sample evaluation and require minimal access to internal model states, making them highly flexible across models and tasks.

In this paper, we demonstrate a critical limitation of existing confidence-based VF-TTS methods: they suffer catastrophic collapse on complex tasks. We observe an interesting phenomenon—uniformly high confidence often signals exploration failure, biasing the model toward confidently wrong answers. To address this, our core insight is that robust epistemic search requires a specific confidence trajectory: the model should perform exploratory branching at the start (manifested as low initial confidence) and converge to a high-confidence final solution.

To realize this insight, we introduce Consilience, a novel selection framework that explicitly evaluates the temporal asymmetry of confidence during reasoning. We operationalize it through a composite metric that actively penalizes high initial confidence while strictly requiring high final certainty.

Extensive experiments on graduate-level mathematics problems and free-form code generation show that Consilience effectively outperforms existing baselines, validating our novel perspective on completion confidence.

Key Points

Problem: Verifier-free test-time scaling (VF-TTS) is essential when external verifiers are unavailable, but confidence-based VF-TTS methods can catastrophically fail on complex reasoning tasks.
Failure Mode: Uniformly high confidence across rollouts indicates that the model has failed to explore, producing confidently incorrect answers.
Core Insight: A desirable reasoning trajectory combines low initial confidence (exploration) with high final confidence (commitment).
Method — Consilience: A selection framework scoring the temporal asymmetry of confidence; penalizes high initial confidence and requires strong final certainty.
Empirical Validation: Outperforms existing confidence-based baselines on graduate-level math benchmarks and free-form code generation.
Practical Advantage: Near-zero sample-evaluation overhead and minimal dependence on internal model state, ensuring broad applicability across models and tasks.