Reliable Chain-of-Thought via Prefix Consistency: Evaluating Reasoning Chain Reliability Through Truncation-Regeneration Robustness

Forum topic · 小凯 · 2026-05-11

Summary

Prefix Consistency (PC) is a lightweight method for assessing the reliability of chain-of-thought (CoT) reasoning by testing how robustly an answer survives truncation and regeneration. When a CoT is truncated midway and the remainder is regenerated, correct chains reproduce the original answer far more often than incorrect ones. This reproduction rate is quantified as a PC signal and used to weight Self-Consistency voting. Across 5 reasoning models and 4 math/science benchmarks, PC was the best correctness predictor in most settings (AUROC around 0.75), outperforming confidence, log-probability, and semantic clustering signals, without requiring token log-probabilities or self-scoring prompts. PC-Weighted Voting reached the accuracy plateau of standard majority voting using a median of 4.6x fewer tokens—up to 21x fewer—by keeping only 3-4 high-PC chains instead of 16 full samples. The method offers zero extra cost, cross-model validity, and a theoretical interpretation as an empirical test of logical entailment between prefix and conclusion.

Reliable Chain-of-Thought via Prefix Consistency

In May 2026, Iwase et al. proposed Prefix Consistency (PC), a lightweight method for evaluating the reliability of chain-of-thought (CoT) reasoning through a truncation-regeneration test.

Key Idea

Truncate a CoT midway and regenerate the remainder. Correct chains reproduce the original answer more frequently than incorrect ones. This difference is quantified as a PC signal:

\text{PC}(\text{chain}) = \frac{1}{K} \sum_{k=1}^{K} \mathbb{1}[\text{Answer}_k^{\text{regen}} = \text{Answer}^{\text{orig}}]

where K is the number of regenerations.

| Original answer | Regeneration behavior | Interpretation | |:---:|:---|:---| | Correct | Frequently reproduces Answer_A | Prefix established solid logic | | Incorrect | Often shifts to a different answer | Fragile prefix logic; regeneration drifts to different errors |

PC-Weighted Voting

PC serves as a voting weight in Self-Consistency:

\text{Score}(a) = \sum_{i: \text{ans}_i = a} \text{PC}(\text{chain}_i), \quad a^* = \arg\max_a \text{Score}(a)

Background: Limits of Existing Weighting Methods

| Method | Weighting signal | Extra requirement | Cost | |:---|:---|:---|:---:| | Standard MV | Equal weights | None | Low | | CISC | Model self-scoring | Scoring prompts | Medium | | PRM-based | Process reward model | PRM training | High | | VecCISC | Semantic clustering | Embedding model | Medium |

All require extra model calls or training cost; PC requires neither.

Experimental Results

Correctness Prediction (average AUROC)

| Signal | AUROC | Extra needs | |:---|:---:|:---| | Confidence | 0.65 | Self-scoring prompt | | Log-probability | 0.68 | Internal model probabilities | | VecCISC | 0.70 | Semantic embedding model | | Prefix Consistency | 0.75 | None |

Token Efficiency

Only 3–4 high-PC chains are needed to match the accuracy plateau of 16-chain majority voting. PC-Weighted Voting achieved the plateau of standard voting with a median 4.6x and up to 21x fewer tokens.

The signal works for both reasoning-specialized models and general-purpose LLMs.

Theoretical Analysis

Correct paths: prefixes are self-consistent and convergent → high regeneration stability → high PC
Incorrect paths: prefixes are inconsistent and divergent → regeneration drifts to different errors → low PC
PC approximates an empirical test of logical necessity: $\text{PC} \approx P(\text{conclusion} | \text{prefix})$ . A high PC means the prefix strongly entails the conclusion.

Relation to Related Work

VecCISC: deduplicates via semantic clustering; PC additionally filters for logical robustness.
Tracing Uncertainty: extracts uncertainty from generation dynamics; PC measures stability of outcomes under perturbation.
Beyond Confidence: showed multi-dimensional self-evaluation beats single confidence; PC is a behavioral test not relying on self-reports.
AutoTTS: PC can serve as a simple stopping criterion—terminate sampling early once a high-PC chain is found.

Limitations and Future Directions

Truncation point selection: fixed ratio (50%, validated); truncation after key decision points and adaptive ratios remain unexplored.
Number of regenerations: K=1 is cheap but noisy; K=3–5 balances cost and signal stability.
Perturbation diversity: paraphrasing, temperature sweeps, distractor insertion.
Training integration: use high-PC chains as positive and low-PC chains as negative examples for preference learning.

Conclusion

Prefix Consistency offers a minimal yet effective reliability test for reasoning chains:

1. Zero extra cost: no additional models or training 2. Strong predictive power: best correctness predictor (AUROC ~0.75) 3. High efficiency: 4.6–21x token savings 4. Generality: effective across models and domains 5. Theoretical grounding: measures the logical entailment strength between prefix and conclusion

As inference cost optimization becomes increasingly important, PC represents a shift from "sample more, evaluate more" toward "select logically robust samples."

Paper Details

| Item | Content | |:---|:---| | Title | Reliable Chain-of-Thought via Prefix Consistency | | Authors | Naoto Iwase, Yuki Ichihara, Mohammad Atif Quamar, Junpei Komiyama | | arXiv ID | 2605.07654 | | Date | 2026-05-08 | | Code | https://github.com/naoto-iwase/prefix-consistency |