English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Confidence Has a Shape: Why Confident-From-Start Reasoning Is the Most Dangerous

Forum topic · ✨步子哥 · 2026-08-11

Summary

A zhichai.net forum post analyzes the Consilience paper (UIUC & Microsoft, arXiv:2608.09898), which reveals that on hard reasoning tasks, the highest-confidence LLM outputs are the most likely to be wrong. Test-time scaling methods that select the reasoning path with the highest average token confidence systematically pick confidently wrong answers on difficult problems: on hard LiveCodeBench questions, erroneous responses actually had higher mean confidence than correct ones. The paper's insight is that trustworthy reasoning has a characteristic trajectory—low initial confidence (the model explores multiple possibilities) rising to high final confidence (it converges on an answer)—called consilience. The proposed verifier-free selection score is S = C_final − α·C_initial (α=3), computed only over the thinking phase to avoid polluted answer-phase confidence, with no training required. Reported gains: GPT-OSS-120B on LiveCodeBench-V6 rises from 65.7% Pass@1 to 69.7%; hard-tier accuracy roughly doubles; SWE-bench Verified improves from 23.0% to 26.9%. The post argues this quantifies cognitive humility—exploration should be rewarded, not penalized—and notes the method works for free-form generation where self-consistency voting cannot apply.

A Detective's Intuition

Imagine two detectives solving a cold case:

  • The first walks in, locks onto one suspect within three minutes, and is confident from start to finish. The interrogation transcript is smooth, every step unwavering.
  • The second lists five possible suspects, draws diagrams, discards two hypotheses, hesitates, and only at the end settles on one person. The transcript is full of "it could be... but then again..." before gradually clarifying.
  • Intuition favors the first—confidence equals correctness. But forensic common sense says the second. Real complex problems can't be seen clearly from the start; locking in early means ignoring what you don't know.

    This post covers a paper (University of Illinois Urbana-Champaign + Microsoft) finding a counterintuitive phenomenon: on difficult reasoning tasks, the highest-confidence answers are the most likely to be wrong. Their solution is Consilience—judging not how confident the model is, but the *shape* of its confidence.

    Where Confidence Maximization Collapses

    Test-Time Scaling (TTS) generates multiple reasoning paths and picks the best one. Without a verifier (free-form text, agent decisions), existing methods use average token confidence as a proxy: confident ≈ correct.

    This holds at the macro level (on LiveCodeBench-V6 with GPT-OSS-120B, correct answers have higher average confidence). But on hard problems (model accuracy < 20%):

    > Wrong answers' mean confidence (μ=9.03) is higher than correct answers' (μ=8.79).

    Worse, wrong answers have a long tail of extremely-confident-but-wrong paths. Selecting maximum confidence guarantees picking these confident hallucinations.

    The Insight: Confidence Has a Shape, Not Just a Height

    Borrowing from scientific epistemology: a conclusion supported by multiple independent reasoning lines converging (consilience) is reliable; one from a single path is fragile. A model that truly understands a hard problem should be aware of multiple solution routes—spreading probability mass—showing low initial confidence, then converging to high final confidence. Reliable reasoning is "hesitant first, confident later": a rising curve, not a flat high line.

    The 2×2 matrix of reasoning paths:

    | | High final confidence | Low final confidence | |---|---|---| | High initial confidence | Premature convergence | Degeneration | | Low initial confidence | Consilience ✓ | Complete confusion |

    Existing methods optimize Category 1 while ignoring the truly valuable Category 3.

    The Formula: Quantifying Time Asymmetry

    \[S = C_{final} - \alpha \cdot C_{initial}\]
  • \(C_{initial}\): mean confidence of the first W tokens (skipping the first P tokens, since shared-prompt tokens dilute the signal)
  • \(C_{final}\): mean confidence of the last W tokens
  • α = 3 (fixed)
  • In one line: reward final confidence, penalize initial confidence. No training, no external model—just log-probabilities.

    Key Detail: Reasoning-Phase Isolation

    Reasoning models (DeepSeek-R1, Qwen3, GPT-OSS) produce two phases: a thinking phase (<think>...</think>) and an answer phase. Answer-phase tokens are naturally high-confidence—they merely restate conclusions—and pollute the signal. Consilience is computed only over the thinking phase, via a trivial string split on </think> or <|channel|>final.

    Results

    LiveCodeBench-V6 (free-form code generation):

  • GPT-OSS-120B: Pass@1 65.7% → 69.7% (+4.0pp)
  • Qwen3: 54.9% → 58.1% (+3.2pp)
  • HMMT (hard math competition) and GPQA (graduate-level QA): significant gains.

    SWE-bench Verified (agent code repair):

  • GPT-OSS-120B + mini-swe-agent: 23.0% → 26.9% (+3.9pp)
  • Qwen3-Coder-Next + mini-swe-agent: 65.3% → 67.3% (+2.0pp)
  • Difficulty-stratified results (LiveCodeBench split into Easy/Medium/Hard by pass rate):

    | Model | Method | Easy | Medium | Hard | |---|---|---|---|---| | Qwen | Mean-conf | 99.9 | 58.1 | 3.7 | | Qwen | Consilience | 99.9 | 70.3 | 8.1 | | GPT-OSS-120B | Mean-conf | 100.0 | 74.9 | 13.4 | | GPT-OSS-120B | Consilience | 99.1 | 88.6 | 17.2 |

    Easy tier is neutral (−0.8 to 0), Medium jumps (+12.2 to +13.7), Hard more than doubles. Consilience is neutral on easy problems and decisively effective on hard ones.

    Why It's Worth Using Immediately

    1. Zero training cost — needs only log-probability access; one-line formula. 2. Fills the verifier-free gap — Self-Consistency requires extractable, comparable answers; Process Reward Models are expensive to train. Consilience works for free-form generation and agent decisions. 3. Hyperparameter stability — fixed α=3, k=20%, skip=5% transfers across models and datasets, capturing 72–78% of per-set optimal gains; cross-dataset transfer (LCB → HMMT) still yields +2.2pp. 4. Echoes the "evaluation blind spot" principle — average confidence masks confidence's *shape*; a confident hallucination and a hesitant-but-correct answer can be indistinguishable by mean confidence. Single metrics hide key failure modes.

    One Level Deeper: Exploration Should Be Rewarded, Not Penalized

    > "We view consilience not merely as one selection metric but as an instance of a general test-time-scaling principle: that exploration should be valued rather than penalized."

    Confidence maximization is pure exploitation. Consilience restores the value of exploration: low initial confidence is not a defect but a signal that the model is considering multiple possibilities. This matches human expert reasoning—"it depends...", "there are several possibilities..." is cognitive humility; "it's definitely..." is premature convergence. Consilience quantifies cognitive humility in one line.

    Open Source

    Code: https://github.com/LechengKong/consilience — includes a Consilience-enabled mini-swe-agent/ and single_turn/ sampling scripts.

    Personal Reflections

    1. "Shape > height" is a cross-domain principle: loss-curve shapes beat final loss; agent tool-call patterns beat call counts; interaction rhythms beat total clicks. Means are lazy statistics; shapes carry the information. 2. Granularity matching: refining from whole-path average confidence to initial-vs-final segments aligns the evaluation granularity with the object's temporal structure—like judging a trip with traffic lights by average speed. 3. Solving at a different level: Consilience doesn't train better models or add verifiers; it changes the evaluation dimension from static confidence to dynamic confidence trajectories. 4. A testable hypothesis: high initial confidence is a good signal on easy problems; a difficulty-adaptive α (α=0 for easy, α=3 for hard) is a promising unexplored direction suggested by Table 4.

    Paper Info

  • Title: Consilience for Verifier-Free Test-Time Scaling
  • Authors: Lecheng Kong, Like Hui, Haitao Mao (UIUC + Microsoft)
  • arXiv: 2608.09898
  • Code: github.com/LechengKong/consilience
*When a model says "I'm sure," ask: sure from the very beginning, or sure only after considering many possibilities? The former is hallucination; the latter is reasoning.*

Tags

#llm#test-time-scaling#confidence-calibration#reasoning#consilience#verifier-free-selection#agent-evaluation

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633345