Echo Chamber Monologue: Why Asking the Same AI 100 Times Won't Reveal the Truth

> Paper: Stochastic Sampling is Epistemically Shallow: The Dimensionality Gap Between Temperature Variation and Model Diversity in LLMs > Author: Izhar Ali (Rowan University) > arXiv: 2607.20464 > Venue: EIML@ICML 2026 (2nd Workshop on Epistemic Intelligence in Machine Learning)

🎲 Opening: One Million Dice Rolls

Imagine consulting a "wise man" about a hard puzzle. Unsure of his answer, you ask him the same question 100 times, asking him to take slightly different reasoning paths each time. The 1st and 51st answers agree on 60% of items, so you feel reassured — "if he repeats himself, he's probably right."

But all 100 answers come from the same mind, the same knowledge base, the same blind spots. If he holds a systematic misconception, you'll get a million variants of the same error.

This is the sharp question Ali poses: **when you sample an LLM 100 times, is the diversity you get diversity of *knowledge*, or just diversity of *noise*?

🧩 Self-Consistency: A Seemingly Clever Shortcut

Self-consistency (Wang et al., 2023) samples multiple answers for a hard problem and takes a majority vote — cheap (one model) and effective (large gains on benchmarks like GSM8K). Derivatives include answer-repetition confidence, semantic entropy, and hallucination detection via sampling disagreement.

All rest on one assumption: variation across samples of the same model carries information about what the model knows and doesn't know.** Ali asks whether that variation carries *structured* knowledge information or is merely surface noise.

🏔️ Two Kinds of Uncertainty

• Per-question uncertainty: for one question, do multiple samples disagree? Self-consistency does this well — like repeatedly flipping a coin to estimate its bias.
• Cross-question structure: across a set of related problems, are errors correlated? A student weak at spatial reasoning fails *all* spatial problems systematically. This reveals capability dimensions — "I'm bad at this *kind* of problem."

Ali's core question: can repeated sampling reveal this cross-question structure?

🔬 Random Matrix Theory: A Mathematical Microscope

Ali borrows Marchenko-Pastur (MP) random matrix theory. Arrange results as a matrix (rows = attempts, columns = questions; 0/1 correctness). For a purely random matrix, the eigenvalue spectrum follows the MP distribution; eigenvalues beyond its boundary indicate non-random structure — a "noise floor" against which real signal stands out.

📊 The Experiment: 100 Samples vs. 24 Models

• Setup A (within-model): one LLM (Qwen2.5-7B), 100 runs per question (temperature=1), producing a 100×500 matrix.
• Setup B (cross-model): 24 different LLMs, one deterministic run each (temperature=0), producing a 24×500 matrix.

Results are striking. Setup A shows at most one eigenvalue above the noise floor, and it is very weak (TW z-score ≤ +0.05, within noise). Setup B shows four clear eigenvalues beyond the noise boundary, verified against a difficulty-matched independent Bernoulli null (500 Monte Carlo draws), which yields at most one signal eigenvalue.

This is the paper's dimensionality gap:

• Within-model dimension: ≤1
• Cross-model dimension: ≥4

🌊 Temperature Dilution: Why Sampling Is "Shallow"

Temperature is a uniform rescaling of logits: it flattens or sharpens the entire output distribution but doesn't distinguish domains, question types, or "I know this" vs. "I'm guessing." Like a lamp uniformly lighting a room, it casts no shadows that reveal structure. Temperature-induced sampling variation is essentially unstructured noise, not structured knowledge difference. Sampling from one model is like wandering different corridors of the same library; using different models means visiting entirely different libraries.

🎯 Locally Committed, Globally Incoherent

When Qwen2.5-7B gets a borderline question wrong, 87.4% of the 100 samples give the same wrong answer — strong "local commitment." But these commitments show no correlated structure across questions: errors are isolated events, not symptom clusters. This contrasts with human cognition, where correlated errors enable diagnosis of systematic weaknesses.

💡 Implications for AI Engineering

1. Selective prediction: self-consistency is fine for per-question uncertainty ("should I abstain?"), but useless for finding *classes* of weakness — that requires genuine model diversity. 2. Model ensembles: diverse models don't just reduce variance; they reveal epistemic dimensions unreachable within a single model. A 24-model ensemble is not "a more expensive version of 100 samples" — it's a fundamentally different information source. 3. Hallucination detection limits: semantic-entropy-style methods remain valid for per-question uncertainty, but cannot expose deeper cross-question systematic hallucination patterns.

🌌 Philosophical Coda: What Does It Mean to "Know"?

Consistency across samples may be mere local inertia, not understanding. Genuine knowing would imply predictable structure across related problems — systematic, diagnosable errors rather than random ones. The dimensionality gap suggests current LLM sampling behavior may resemble memory rather than understanding.

🔮 Conclusion

Self-consistency is not a cheap substitute for ensembles — it is a fundamentally different, informationally impoverished approximation. Warnings: don't over-trust sampling's surface diversity. Direction: when pursuing true epistemic dimensionality, model diversity may be more valuable than sampling depth. A recording of one instrument played 100 times can never replace the harmony of an orchestra.

📚 References

1. Ali, I. (2026). *Stochastic Sampling is Epistemically Shallow*. arXiv:2607.20464. EIML@ICML 2026. 2. Wang, X., et al. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. *ICLR*. 3. Lakshminarayanan, B., et al. (2017). Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. *NeurIPS*. 4. Marčenko, V.A., & Pastur, L.A. (1967). Distribution of eigenvalues for some sets of random matrices. *Mathematics of the USSR-Sbornik*. 5. Farquhar, S., et al. (2024). Detecting Hallucinations in Large Language Models Using Semantic Entropy. *Nature*.