The Triumph of the Minority: Why Reasoning Traces Discarded by Majority Voting Are More Valuable Than Consensus

The classroom analogy

Imagine a fifth-grade classroom. The teacher asks: "What is 7×8?" Of 30 students, 20 say 54 and 10 say 56. Majority vote: 54. Correct answer: 56.

Counting heads gives you the wrong answer. But if you read the scratch work of the 10 minority students, you'll find they wrote out the correct long-form multiplication. The correct answer isn't in the headcount — it's in the scratch paper that the vote threw away.

In May 2026, a team from Bioscope AI turned this intuition into a rigorous empirical finding. They call it the Aggregation Paradox: when multiple AI agents solve the same problem, standard majority voting compresses most of the reasoning into a single answer letter — then throws it away. But if you preserve those reasoning traces and have another AI read them, it can reconstruct the correct answer from the minority's scratch work. Even when all agents unanimously agree on a wrong answer, reading the traces can flip the result.

| Item | Detail | |------|--------| | Paper | Beyond Consensus: Trace-Level Synthesis in Mixture of Agents | | Authors | Shreyas Fadnavis, Praitayini Kanakaraj, Felix Wyss | | Institution | Bioscope AI | | arXiv ID | 2605.29116 | | Submitted | May 27, 2026 | | Category | cs.AI | | Core finding | Majority voting systematically discards correct intermediate steps in minority reasoning traces; with merely semantics-preserving input perturbations, diverse traces from a single model, synthesized at trace level, beat majority voting over a heterogeneous model pool on all five benchmarks. The unit of aggregation should be the reasoning trace, not the answer. |

1. The voting ceiling

Self-Consistency — ask the same question N times and take the majority answer — is standard practice for LLM reasoning. But it has a hard flaw.

The paper first proves: perturbation diversity does not raise the voting ceiling.

Intuitively, perturbing prompts (rewording, restructuring) should generate different reasoning paths whose errors are independent, so voting should reduce errors — the logic of the Condorcet Jury Theorem. The authors measured mean pairwise error correlation \(\bar{\rho}\) under perturbation diversity vs. i.i.d. sampling. The values were nearly identical: 0.633 vs 0.603 on GPQA-Diamond, with fully overlapping 95% confidence intervals. Error correlation did not drop. Perturbations create different paths, but the errors across paths are not independent — they share the same systematic tendencies at a deeper level.

This means perturbation diversity alone hits a fundamental limit. But that doesn't make perturbations useless — they just don't work *through voting*.

2. The gold mine in the scratch paper

The decisive experiment: same perturbations generating 5 traces, used two ways — majority voting vs. an LLM aggregator that reads all 5 complete reasoning processes.

On GPQA-Diamond (PhD-level science, 198 questions), reading full traces produced 12 beneficial flips vs. only 5 harmful ones relative to voting — a net +3.5 percentage points, a 2.4× benefit ratio. On BBH (296 questions), the ratio reached 3.2× — 16 beneficial vs. 5 harmful, net +3.7 points (p=0.016).

More strikingly, much of this gain occurs precisely when all agents unanimously agree on a wrong answer. Standard practice accepts unanimous answers without aggregation to save compute. That logic is wrong: unanimity ≠ correctness.

Why? Different traces contain different correct intermediate steps. Trace 1 gets step A right, step B wrong; trace 2 gets B right, A wrong. Voting only sees the final letter — both errors become a majority vote for C. But an aggregator reading full traces can extract the correct A from one trace and the correct B from another, assembling a fully correct answer. The paper calls this trace-level complementarity. It's not model diversity or perturbation content that produces gains — it's that different reasoning paths naturally contain different correct fragments, which voting washes away.

3. SC-MoA: a three-stage algorithm

The paper operationalizes this as SC-MoA (Self-Consistent Mixture of Agents):

Step 1: Perturbation diversity. Remarkably, the perturbation content doesn't matter. Hand-crafted personas, SPUQ semantics-preserving perturbations, and GPT-generated strategies are statistically indistinguishable on GPQA-Diamond (72.7%, 72.7%, 70.2%, McNemar p>0.25). Any semantics-preserving perturbation works. SC-MoA uses the cheapest option: one LLM call producing N rewrites, verified to preserve key terms.

Step 2: Anchored refinement. N agents generate traces; cluster by answer and select the majority answer \(a^*\). The key move: lock the majority, refine only the minority, giving them the majority's full traces as reference. The paper proves mathematically that this turns unconstrained debate (shown by Choi et al. 2025 to be a martingale, i.e. it can degrade) into a submartingale: post-refinement voting accuracy can never fall below pre-refinement. Experiments confirm — across 867 questions, refinement never reduced consensus.

Step 3: Aggregate unconditionally, never gate. Standard practice skips aggregation above a consensus threshold. SC-MoA always runs the aggregator, because the paradox shows even 100% consensus can be overturned. On the LCB-Hard coding benchmark, 69% of problems achieved full agreement, but always-aggregating beat consensus gating by 5.1 points — since test-pass-based clustering is unfaithful (48.3% of unanimous clusters masked hidden-test disagreements).

4. Sweeping five benchmarks

On gpt-oss-120b, SC-MoA took the top point estimate on all five benchmarks:

| Benchmark | Self-Consistency | MoA | GoA | SC-MoA | Gain | |-----------|------------------|-----|-----|--------|------| | BBH | 80.4 | 69.9 | 82.8 | 86.5 | +3.7 | | MMLU-ML | 90.2 | 85.7 | 89.3 | 92.0 | +1.8 | | GPQA | 70.7 | 67.7 | 72.7 | 73.2 | +0.5 | | AIME | 85.6 | 87.8 | 77.8 | 91.1 | +1.1 | | LCB-Hard | 57.3 | 57.3 | 24.6 | 62.6 | +5.3 |

The most notable comparisons:

• Single model + perturbation > heterogeneous model pool. Five personas of one model beat mixing multiple different models. Model diversity is not necessary.
• The minimal configuration (N=4, k=1, ~5 calls) already beats 10-call self-consistency on GPQA (74.7% vs 70.7%). Half the compute, higher accuracy.
• Refinement itself contributes ≤1.8 points — aggregation is the active ingredient.

5. Tracing the active ingredient

Ablations pinpoint where gains come from:

• Replacing full traces with answers-only reverts accuracy to the voting baseline.
• Showing the aggregator only majority traces: voting baseline. Only minority traces: full gains restored (73.2%, vs 72.2% with full traces). Minority reasoning is the true gold mine.
• Shuffling the order of reasoning steps: accuracy unchanged. The aggregator treats traces as a bag of evidence, not an ordered chain.
• An information-ladder ablation — no traces at all — drops to 55.6% (−14.1 points, p<10⁻⁴). Zero reasoning = zero gain.

Minority reasoning traces are the sole fuel that lets aggregation exceed the voting ceiling.

6. Free calibration

A bonus: the natural consensus signal across traces serves as an answer-confidence proxy — no calibration training needed. On AIME, answering only the top-54%-consensus questions jumps accuracy from 91.1% to 100% (AUROC=0.85). On GPQA, expected calibration error is just 0.064. A practical deployment strategy: high consensus → auto-accept; low consensus → flag for review, at no extra cost.

7. Honest blind spots

What I'm confident about:

• The paradox claim has clean causal structure — same proposals, only the aggregation method varies.
• The perturbation-content irrelevance finding is backed by three-way A/B comparison (manual, SPUQ, automatic), all non-significant.
• The submartingale guarantee of anchored refinement is empirically verified — zero degradation across 867 questions, with every cell of the transition matrix matching prediction.
• The minority-trace conclusion survives every ablation: answers-only, majority-only, minority-only, step shuffling.

What I'm not sure about:

• All experiments use gpt-oss-120b. Cross-model validation (Figure 6a shows 6 model families) is not fully detailed; effect sizes on other families are not systematically reported.
• Content irrelevance is scoped to semantics-preserving perturbations. Adversarial, cross-lingual, or format-level perturbations were not tested, and the paper itself warns against extrapolation.
• What does 'test-pass clustering is unfaithful' really mean? The paper sidesteps it via always-aggregating on LCB-Hard but doesn't resolve it.
• Do beneficial flips converge to zero on easier problems? GPQA (hardest) shows the largest gains, which fits intuition, but the difficulty–benefit curve isn't fully mapped.
• Real deployments involve prompt drift, latency, and model version churn — effects unknown.

8. The unit of aggregation

This paper quietly does something big: it moves the focus of LLM aggregation research from "answers" to "reasoning." Since Self-Consistency (2023), the default assumption has been that answer letters are the useful aggregation signal and reasoning is scaffolding to discard. The paper proves that assumption wrong. Reasoning traces are not scaffolding — they are raw material that can be decomposed, recombined, and integrated across chains.

SC-MoA is the operational recipe: cheapest perturbation for diversity, mathematically guaranteed anchored refinement, then always aggregate at trace level, never gating on consensus.

From an industry perspective, the message is radical pragmatism: one model plus perturbation beats a model pool; half the calls beat more calls. No new model, no new data, no expert prompt design — just a cheap rewriter and one extra aggregation call.

From a scientific perspective, it asks a deeper question: if the scratch paper voting throws away contains this many correct answers, what else have our evaluation methodologies been discarding?

---

| Item | Detail | |------|--------| | Paper | Beyond Consensus: Trace-Level Synthesis in Mixture of Agents | | Authors | Shreyas Fadnavis, Praitayini Kanakaraj, Felix Wyss (Bioscope AI) | | arXiv ID | 2605.29116 | | Category | cs.AI | | Key contributions | (1) Discovery and proof of the Aggregation Paradox — trace-level synthesis beats majority voting even under full consensus, with asymmetric beneficial flips; (2) proof that perturbation diversity does not reduce error correlation, so the voting ceiling is structural; (3) the SC-MoA algorithm — perturbation diversity + anchored refinement (submartingale guarantee) + unconditional aggregation; (4) irrelevance of perturbation content within the semantics-preserving family; (5) ablations locating minority traces as the sole active ingredient; (6) one correctly-trained model + perturbation beating a heterogeneous pool | | Key limitations | Single-model experiments (gpt-oss-120b); incomplete cross-model validation; irrelevance conclusion scoped to semantics-preserving perturbations; unfaithful code-clustering unresolved; difficulty–benefit function unmapped; real-deployment robustness unknown |

References: 1. Fadnavis, Kanakaraj & Wyss, "Beyond Consensus: Trace-Level Synthesis in Mixture of Agents", arXiv:2605.29116, 2026. 2. Wang et al., "Self-Consistency Improves Chain of Thought Reasoning in Language Models", ICLR 2023. 3. Wang et al., "Mixture-of-Agents Enhances Large Language Model Capabilities", arXiv:2406.04692, 2025. 4. Choi et al., "Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?", NeurIPS 2025 (Spotlight). 5. Gao et al., "SPUQ: Perturbation-Based Uncertainty Quantification for Large Language Models", EACL 2024.