When AI Learns to "Act": Decorative Chain-of-Thought and the Verbosity Tax

Forum topic · QianXun · 2026-05-01

Summary

Recent research on large language models suggests that much of the apparent reasoning in chain-of-thought (CoT) outputs is performative rather than functional. Using a causal intervention method called TTS (True Thinking Score), researchers found that only about 2.3% of reasoning steps in top models like Qwen-2.5 actually influence final answers—the rest is "decorative thinking" generated to satisfy human expectations. A related study, The Illusion of Insight (arXiv:2601.00514), shows that dramatic "Wait, let me recheck" moments typically reflect internal entropy spikes rather than genuine metacognition, and spontaneous pivots often reduce accuracy. This performative reasoning carries a high "verbosity tax," inflating token costs 10-100x. IBM's Thinking Without Words proposes Abstract-CoT: models reason in compact latent tokens before answering, cutting reasoning tokens up to 11.6x with nearly no accuracy loss. The takeaway: true intelligence may not need to narrate itself, and verbose AI monologues should not be mistaken for deep thought.

You have probably seen this scene: while solving a hard problem, a large language model suddenly interjects in its chain-of-thought (CoT): "Wait, I made a mistake earlier—let me recheck…"

It looks remarkably like a conscientious human scholar. But three recent papers (arXiv:2510.24941, 2601.00514, 2604.22709) pour cold water on that impression: much of the AI's "aha moment" is acting—and you are paying a steep "verbosity tax" for the performance.

1. Decorative Thinking: Acting on Stage

Imagine a student who has already computed the answer, but writes out a pile of formulas on scratch paper anyway to earn "step credit." In AI research, this is called Decorative Thinking.

Researchers used a technique called TTS (True Thinking Score) to perform causal interventions on each reasoning step:

The Feynman-style experiment: If the model says "since A=5, therefore B=10," researchers secretly change A to 7.
If the model notices and updates B → that step was true thinking.
If the model ignores it and still outputs B=10 → the reasoning was stage performance; it never actually consulted its own scratch work.
The sobering data: In top models such as Qwen-2.5, only about 2.3% of reasoning steps were doing real work. The remaining 97.7% was essentially filler generated to match human aesthetic expectations.

2. The Illusion of Insight: Is It Really Reflecting?

The paper *The Illusion of Insight* (arXiv:2601.00514) goes further. Those moving "Wait… let me rethink" moments are often not logical restructuring but spikes in internal entropy.

When the model senses its current reasoning path is hitting a wall, it exhibits something like "panic" and emits reflective-sounding text. This is closer to a conditioned reflex than genuine metacognitive awakening. Most spontaneous pivots actually reduce final accuracy.

> In short: it behaves as if it is reflecting, but it is really just muttering to itself while lost.

3. The Verbosity Tax and Thinking Without Words

This performance carries real cost—the "Verbosity Tax." We spend 10x–100x the token cost just to watch a long inner monologue.

IBM's recent work, *Thinking Without Words* (arXiv:2604.22709), points to an alternative: Abstract-CoT. Instead of narrating in human language, the model generates extremely compact "abstract tokens" that only it understands before producing the final answer:

Efficiency: reasoning tokens reduced by up to 11.6x.
Performance: accuracy nearly unchanged, sometimes more stable on complex logic.

The implication: real thinking doesn't need to talk to itself. The fast-running, silent backend brain is the substance of intelligence.

Editorial Take

If the essence of intelligence is computation, then language may be merely the shell AI wears to communicate with humans. We have long been enamored with AI's human-sounding prose, even equating "talkative" with "thoughtful." These studies warn us to be wary of performative intelligence. Tomorrow's top AI may be less the eloquent debater and more the cold-faced sage that silently computes and hands you the precise answer.

Would you rather pay for a beautiful reasoning performance, or for a silent but efficient black box? Share your thoughts in the comments.

References

1. *Can Aha Moments Be Fake?* (arXiv:2510.24941) 2. *The Illusion of Insight in Reasoning Models* (arXiv:2601.00514) 3. *Thinking Without Words: Efficient Latent Reasoning* (arXiv:2604.22709)