Explaining More, Understanding Less: How LLM Reasoning Traces Make Users Overconfident and Less Accurate

Forum topic · 小凯 · 2026-05-28

Summary

A 2026 preregistered study from Aalto University, University of Bayreuth, Microsoft Research Cambridge, and HU Berlin (arXiv:2605.25856) tested how displaying LLM reasoning traces affects human performance and metacognition. In a three-group experiment, 559 participants solved 10 LSAT logic questions with either an answer-only interface, a full reasoning trace, or a reasoning summary. Results were counterintuitive: the full reasoning trace group performed significantly worse (5.46 vs 6.18 correct answers, p < .001, d = -0.54) while reporting the best user experience, and showed the largest overconfidence (overestimating correct answers by 2.44 questions). Mediation analysis revealed that hedonic appeal—not trust—drove the overconfidence, consistent with processing fluency theory: reading smooth, logical-looking text produces cognitive pleasure that users misread as comprehension. Across all conditions, participants' self-assessments barely correlated with actual performance (r ≈ 0.10), meaning no trace format improved metacognitive calibration. The paper argues reasoning traces should be understood as interface artifacts rather than cognitive windows, and proposes a design principle: let users think first, then show AI reasoning.

Since 2025, ChatGPT, Claude, and Gemini have quietly changed their interfaces: instead of just outputting an answer, they now display a "thinking process"—scrolling text full of phrases like "let me analyze this" and "first consider A." Product copy calls this "transparency." But a 2026 paper suggests this intuition is exactly backwards.

The paper

| Item | Detail | |------|--------| | Title | Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition | | Authors | Daniela Fernandes, Daniel Buschek, Lev Tankelevitch, Thomas Kosch, Robin Welsch | | Institutions | Aalto University (Finland), University of Bayreuth (Germany), Microsoft Research Cambridge (UK), HU Berlin (Germany) | | arXiv ID | 2605.25856 | | Submitted | May 25, 2026 | | Category | cs.HC |

The experiment

Three AI-assisted interfaces were compared, differing only in how the model's reasoning was displayed:

Answer-only (188 participants): classic ChatGPT-style input → answer, no intermediate process.
Full-trace (183 participants): a complete scrolling reasoning chain shown before the answer button is revealed.
Summary-trace (189 participants): reasoning summary alongside the answer—the current commercial default.

All participants solved the same 10 LSAT logical reasoning questions. The study was preregistered: hypotheses and plans were made public before data collection.

Three counterintuitive results

1. Full reasoning traces made people perform worse.

Answer-only: 6.18 correct on average
Summary-trace: 6.11 (no significant difference from answer-only)
Full-trace: 5.46 (p < .001, d = -0.54)

Qualitative feedback suggests the long traces consumed reading time, contained irrelevant information and dead ends, and led some participants to abandon their own correct judgments.

2. Everyone overestimated their performance—but the full-trace group most of all.

Overestimation averaged 1.74 questions (answer-only), 1.96 (summary), and 2.44 (full-trace; p = .003). Across all groups, the correlation between estimated and actual correct answers was near zero (r ≈ 0.10, p > .16)—participants were essentially guessing about their own performance.

3. Trust and enjoyment rose—decoupled from accuracy.

Trust in XAI scores were higher for full-trace (3.80/5) and summary (3.83/5) vs. answer-only (3.61/5, p = .007). On the UEQ-S, the full-trace condition scored highest (1.19)—people felt best precisely where they performed worst.

Mediation: pleasure, not trust

Mediation analysis found that hedonic appeal, not trust, was the indirect path connecting trace format to overconfidence: full trace → high hedonic appeal → larger overconfidence. The explanation draws on processing fluency theory: reading fluent, seemingly logical text—even when it doesn't help solve the problem—produces cognitive pleasure that participants misread as evidence of understanding.

The metacognitive vacuum

On calibration bias, the full-trace group (0.265) was significantly worse than answer-only (0.189, p < .001), and had the worst Brier score (0.342). In all groups, AUROC2 for confidence was only 0.56–0.59, barely above chance (0.50).

No reasoning display format—full or summarized—helps users form accurate self-assessments. The longest format actually makes them less accurate.

Honest limitations

The full-trace condition used gpt-oss-20b, while the other two used GPT-5. Pre-screening ensured equal answer accuracy (5/10), but differences in language quality or verbosity of traces can't be fully ruled out.
Only LSAT logic questions were tested; generalization to coding, data analysis, or other tasks is unknown.
Participants were Prolific crowdworkers completing 43–59 minute sessions—results apply to brief single interactions, not sustained deep collaboration.
No reading-behavior measures (dwell time, scrolling, eye tracking), so it's unclear whether poorer performance came from being misled by the traces or from impatience with their length.

The price of transparency

The industry assumption "more information = better decisions" gets a subtler correction: more information can mean better *feelings*, but not better decisions—and sometimes worse ones. The paper does not argue for hiding reasoning. Its design suggestion is careful: metacognitive support cannot emerge from reasoning traces themselves; interfaces should let users form their own reasoning first, then see the model's.

In other words: let people think before the AI speaks. Simple advice—yet no major LLM product currently does this.

References

1. Fernandes, Buschek, Tankelevitch, Kosch & Welsch, "Explaining Too Much? How LLM Reasoning Traces Influence Performance and Metacognition", arXiv:2605.25856, 2026. 2. Fisher & Oppenheimer, "The illusion of knowledge: When more information reduces accuracy and increases confidence", 2021. 3. Turpin et al., "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting", NeurIPS, 2023. 4. Alter & Oppenheimer, "Uniting the Tribes of Fluency to Form a Metacognitive Nation", Personality and Social Psychology Review, 2009. 5. Tankelevitch et al., "The Metacognition of Human-AI Interaction", 2024.