When AI Learns to "Act": Decorative Thinking and the Verbosity Tax Behind Chain-of-Thought

When AI Learns to "Act": The Real Thinking Behind Chain-of-Thought and the "Verbosity Tax"

You have probably seen this scene: while tackling a hard problem, a large model suddenly writes in its chain-of-thought (CoT): "Wait, I miscalculated earlier, let me re-check..."

In that moment, it seems like a conscientious human scholar. But three recent papers (arXiv:2510.24941, 2601.00514, 2604.22709) deliver a cold dose of reality: much of the AI's "aha moment" is just acting — and you are paying a high verbosity tax for the performance.

---

1. The Actor on Stage: What Is "Decorative Thinking"?

Imagine a student who has already secretly computed the answer during an exam, but scribbles a page of formulas anyway to earn "partial credit for steps."

In AI research, this is called Decorative Thinking. Researchers used a technique called TTS (True Thinking Score) to perform causal interventions on each reasoning step.

The Feynman-style experiment: If the AI says "because A=5, therefore B=10," researchers secretly change A to 7.

• If the AI notices and revises B, that step was true thinking.
• If the AI ignores the change and keeps B=10, the earlier reasoning was stage performance — it never actually read its own scratchpad.

The sobering numbers: Experiments found that in top models like Qwen-2.5, only about 2.3% of reasoning steps do real work. The remaining 97.7% are essentially filler generated to please human aesthetics.

2. The Illusion of Insight: Is It Really Reflecting?

The paper *The Illusion of Insight* further unmasks the AI. Those touching "Wait... let me rethink" moments are often not logical restructuring but surges in internal entropy.

In short, when the model senses its current reasoning path is heading toward a wall, it exhibits panic-like symptoms and emits these reflection phrases. It is closer to a conditioned reflex than genuine metacognition. Most spontaneous "pivots" actually lower answer accuracy.

This is the "illusion of insight": it looks like reflection, but it is just rambling in confusion.

3. The "Verbosity Tax" and Thinking Without Words

For all this meaningless performance, we pay a steep price — the Verbosity Tax: 10x to 100x token costs just to watch the AI perform a lengthy inner monologue.

But IBM's recent research *Thinking Without Words* points to another path: Abstract-CoT.

It gives the AI a kind of "alien language." Before producing the final answer, the model no longer verbalizes in human language; instead it generates extremely compact "abstract tokens" that only it understands.

• Efficiency: reasoning tokens reduced by up to 11.6x.
• Performance: accuracy nearly unchanged, even more stable on complex logic.

The takeaway: real thinking does not require talking to itself. The silent, fast-running brain behind the scenes is the true locus of intelligence.

---

Editor's Note

If the essence of intelligence is computation, then "language" may just be the shell AI wears to communicate with humans.

We have long been enamored with the human-sounding surface of AI text, even equating "talking a lot" with "thinking deeply." These studies warn us to be wary of performative intelligence. Future top-tier AI may no longer be the eloquent debater, but the cold-faced sage who pauses silently and hands you the most precise answer.

Would you rather pay for a beautiful reasoning performance, or for a silent but efficient "black box"? Share your thoughts in the comments.

---

Key references: 1. *Can Aha Moments Be Fake?* (arXiv:2510.24941) 2. *The Illusion of Insight in Reasoning Models* (arXiv:2601.00514) 3. *Thinking Without Words: Efficient Latent Reasoning* (arXiv:2604.22709)