Share, Don't Repeat: How CPT Breaks the 'Information Silos' in Parallel Reasoning

The original post (in Chinese) explains CPT with a vivid metaphor: parallel reasoning today is like sending ten people into the same maze, each with a blank map, all exploring from scratch. CPT says: pin your discovered map fragments to a public bulletin board so newcomers can see them immediately. No training, no model changes—just broadcast what is already known.

1. The Information Silos of Parallel Reasoning

For complex reasoning (math, coding, science), a simple acceleration idea is Test-Time Scaling (TTS): allocate more compute at inference to explore more solution paths. Two main routes exist:

1. Sequential scaling: one long chain of thought with backtracking and self-correction (e.g., Long CoT as in DeepSeek-R1). 2. Parallel scaling: multiple branches explore simultaneously, then vote or pick the best (Best-of-N, Self-Consistency, Tree-of-Thought).

Parallel scaling is naturally parallelizable—wall-clock latency depends only on the longest branch. But it has a hidden cost: information silos. Branches cannot tell each other "that path is a dead end" or "x must be even," so they repeatedly:

• Walk into dead ends already verified by other branches
• Re-derive constraints others have found
• Re-collect the same "basic facts" instead of building on them

The paper shows redundancy grows over time: the more branches and the longer they run, the more duplicated discoveries. Ten independent branches may have an effective information width of only 3–4 independent branches.

2. CPT's Three Mechanisms

2.1 Information Extraction

Branches cannot share raw outputs (too long, too noisy, would overflow context). Instead, after each branch generates a chunk of C tokens, a policy model π extracts information units from the new content plus history:
• Intermediate conclusions ("we have shown x=2")
• Constraints ("since y must be integer, ...")
• Observations ("substituting gives LHS > RHS")
• Counterexamples ("n=3 leads to contradiction")

Formula:

\[Z^t_i = \text{Extract}_\pi(x, h^t_i, \Delta h^t_i)\]

Key insight: share thinking results, not thinking processes—like sharing meeting minutes, not recordings.

2.2 Deduplicated Information Pool

Ten branches may discover the same fact. CPT maintains a deduplicated query-level information pool P^t: 1. New units enter a candidate pool 2. Text embedding similarity φ(candidate) is compared against all existing entries; if max similarity exceeds τ_dup = 0.75, the candidate is discarded 3. Otherwise it joins the pool

Ten branches finding "x=2" is no more informative than one. The goal is different branches discovering different information—maximizing information density.

2.3 Adaptive Broadcast Scheduling

Two phases:

Probe phase: branches explore independently for several steps; per-step new information units n^t are counted; window-average gain g_j and relative gain r_j = g_j / (g_1 + ε) are computed.

Broadcast phase: when r_j exceeds a threshold, sample up to M=512 items from the pool, serialize them into a shared context block B, and insert B into every branch's input context before the next step. When r_j drops below the threshold, broadcasting stops—diminishing returns.

The wisdom: no broadcasting early (too little information), concentrated broadcasting in the middle (richest output), silence late (marginal gains gone).

3. Full Workflow Example

For an AIME problem with K=8 branches and C=512-token chunks:

• Step 0: all branches see the same problem; pool empty
• Steps 1–3 (probe): independent generation, extraction, deduplication, gain computation
• Steps 4–7 (broadcast): sampled pool content B is injected into each branch's context; branches build on shared conclusions; loop of extraction → dedup → broadcast continues
• Step 8+: stop broadcasting when relative gain is low, or terminate early when a branch reaches an answer
• End: collect all final answers; vote or select best (as in traditional methods)

4. Experimental Results

Evaluated on HMMT and AIME with Qwen3-4B-Thinking and Qwen3-30B-A3B-Thinking, against:

• Base parallel sampling (no sharing)
• DeepConf (confidence-based pruning)
• LeaP (periodic summary sharing)

Findings:

• CPT establishes a better accuracy–latency Pareto frontier on all models and datasets (HMMT24/25, AIME24/25/26), across multiple rollout budgets
• Gains hold for both 4B and 30B models
• Redundant information discovery per step drops significantly, while new-information discovery stays comparable
• CPT is also better on the accuracy-vs-token-consumption frontier
• Extraction and dedup add only moderate overhead; parallel sampling remains the dominant latency source
• Caveat: because broadcasting via input context requires re-prefilling updated contexts, CPT is not always dominant under a pure FLOPs budget
• Information-theoretic view: effective width K_eff = K / (1 + (K−1)ρ), where ρ is cross-path correlation. High ρ gives diminishing returns from more branches; CPT lowers ρ by sharing, raising effective width

5. Limitations and Future Directions

Known limitations: 1. Synchronous protocol: information found at step t is only usable at step t+1, wasting one step of generation 2. Re-prefilling FLOPs: each broadcast requires re-encoding updated context 3. Hyperparameter sensitivity: M=512 and τ=0.75 were tuned; no automatic tuning mechanism

Future directions: 1. Cache-aware sharing: inject pool embeddings as extra KV cache entries, avoiding re-prefilling 2. Attention-level sharing: branches attend directly to a shared "information pool" in attention layers—deeper architecture change, potentially order-of-magnitude efficiency gains 3. Asynchronous broadcasting: real-time pool updates instead of synchronization points

6. Why CPT Matters

The core insight: parallel reasoning branches are not "ten independent people" but "one team." Conventional methods assume diversity comes from independence, yet the paper shows unshared branches converge to similar paths (high ρ). CPT's answer: diversity comes from sharing, not isolation. Sharing exists not to reach consensus but to avoid repetition, freeing resources to cover a larger search space.

Crucially, CPT is completely training-free—no model changes, no fine-tuning, no extra data. Any existing reasoning model can adopt it immediately. As TTS becomes the mainstream paradigm, CPT points to an important direction: compute should not only run more chains—it should teach the chains to talk to each other.

References

1. Wang, X., et al. (2026). Share More, Search Less: Collaborative Parallel Thinking for Efficient Test-Time Scaling. *arXiv:2605.27030*. 2. Snell, J., et al. (2024). Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Model Parameters. *NeurIPS*. 3. Fu, Y., et al. (2025). DeepConf: Confidence-based Pruning for Parallel Test-Time Scaling. *ICML*. 4. Liu, B., et al. (2025). LeaP: Learning from Peers in Parallel Reasoning. *OpenReview*. 5. Yao, S., et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. *NeurIPS*.