The Classroom Experiment
Imagine a math class. A student writes a long, elegant derivation and reaches the correct answer. The teacher is satisfied — until he runs an experiment: ask the student to skip the derivation and write the answer directly.
The result: the student's accuracy writing answers directly is nearly identical to the accuracy with full derivations.
What does this mean? The derivations may not be the "path to the answer" but a "justification written after the answer was already known."
This is exactly what Jingyu Liu et al. (Renmin University + ByteDance) found in the paper *Where Do CoT Training Gains Land in LLM based Agents?* (arXiv:2606.26935). Through a series of careful experiments, they show: the gains from CoT training land mostly outside the CoT.
Core Question: What Does Training Actually Improve?
Chain-of-thought (CoT) reasoning is standard for current LLM agents: the model generates a reasoning chain, then outputs an action. The mainstream narrative says CoT training teaches models "to reason" — longer, more complex chains mean smarter models.
But prior work on "unfaithful CoT" suggests models may already know the answer from the prompt before generating CoT, which is then mere post-hoc rationalization. If true, what does CoT training actually improve: the ability to revise actions via reasoning, or the ability to predict actions directly from the prompt?
The paper runs a clean controlled experiment comparing two decoding modes:
- CoT action: the standard mode, reasoning first, then action.
- Prompt action: an
<action>tag is pre-filled at the start of the response, forcing the model to skip reasoning and output the action directly. - About 80% of attention mass lands on prompt tokens, not CoT tokens.
- Even at top-K attention positions, prompt tokens dominate — not a purely mechanical effect of prompt length.
- Gradient decomposition along the value path shows prompt tokens also receive a larger share of gradient signal.
If CoT training genuinely improved reasoning, the advantage of CoT action over prompt action should grow with training. If training mostly improves direct prediction, accuracy in both modes should rise in parallel.
Three Lines of Evidence: Gains Land on the Prompt Side
Evidence 1: Parallel Improvement
On ALFWorld (household tasks), ScienceWorld (scientific reasoning), and BFCL (function calling), the authors train Qwen3-8B and Llama models with SFT and RL, comparing the two decoding modes across checkpoints.
The result is clear: prompt action and CoT action accuracy rise in parallel. The model improves with and without reasoning by the same amount, and prompt/CoT consistency (frequency of arriving at the same action) also increases. If reasoning were key, its marginal benefit should grow over training. It doesn't.
Evidence 2: The Gap Stays Flat in Online Evaluation
To rule out memorization effects, the authors evaluate online on unseen tasks. Figure 3(a) shows the trajectory success-rate gap between CoT action and prompt action stays flat throughout training. Figure 3(b), using GPT-5.4 as a judge to compare the two actions at the same decision points, shows the CoT-vs-prompt win rate also remains stable.
Training didn't make CoT more "useful." It made both modes better together.
Evidence 3: Conflicting Reasoning Chains Can't Sway Later Checkpoints
The most striking experiment: replace the model's original reasoning chain with one supporting a different action, then see whether the final action follows the prompt or the injected chain.
Result: later checkpoints increasingly keep the prompt-implied action, even when the reasoning chain says otherwise. If training strengthened the reasoning-action causal link, swapping chains should more easily change actions. The opposite happens — the model becomes more "stubborn" about the prompt, and the chain's influence declines.
In the paper's words: later checkpoints are less likely to revise the action in response to CoT.
Why the Prompt Has a Structural Advantage
The authors also give a mechanistic explanation. In agent settings, prompts contain task instructions, interaction history, and environment feedback, and are typically far longer than the CoT. This length asymmetry creates a structural attention hegemony:
A Counterintuitive Intervention: Supervise Actions Less
If the problem is that action supervision mainly reinforces the prompt→action shortcut, the fix follows: mask action-token supervision on a fraction of training samples, optimizing only the CoT span.
Concretely: on a random k% of samples, compute loss only on the CoT portion; the rest keep standard supervision. The authors call this "reduced action supervision," with k = 0.3 working best.
It does two things at once:
1. OOD performance improves on most model-environment combinations in ALFWorld and ScienceWorld (Figure 6). Gains are smaller on BFCL — consistent with the prediction that interventions pay off most where the prompt shortcut is strongest, since BFCL's function-calling prompts are shorter. 2. The CoT-prompt gap reopens (Figure 7), meaning CoT regains some power to change actions.
On Qwen3-8B, SFT + reduced action supervision combined with DPO lifts average performance across the three environments from 0.61 (FRODO baseline) to 0.67 (Table 1).
The intervention is counterintuitive because it deliberately gives up some action supervision — under the intuition that "the action is the final goal," this looks like weakening the objective. But because standard training routes action supervision into the shortcut, giving the model a bit less action supervision actually helps it learn better.
What Does This Mean?
"CoT Training" May Be a Misnomer
If the main gains land on prompt action, the name itself is misleading. More accurately: we are training models to predict actions directly from prompts, and CoT is a byproduct. This closes the loop with unfaithful-CoT research: CoT is unfaithful at inference time (the answer is already known) and at training time (training doesn't improve reasoning itself).
Reasoning Is a Witness, Not a Detective
Analogy: the prompt is all the physical evidence at the crime scene; CoT is the courtroom witness. We thought training the witness would make the detective better at solving cases; in fact the detective increasingly relies on the evidence — the witness merely repeats conclusions the evidence already implies. The more training, the more the witness sounds like a tape recorder, and the less the detective listens (the conflicting-chain experiment).
A Warning for AI Safety
If CoT mostly "shows work" rather than "does work," then CoT-based interpretability and alignment rest on a fragile assumption: that CoT faithfully reflects the model's decision process. This paper adds stronger evidence: CoT isn't just unfaithful at inference; it becomes more decorative after training.
Engineering Takeaway: Less Is More
Reduced action supervision is directly usable. If you're training agents, especially long-context ones, supervising only the CoT on 30% of samples may beat supervising actions throughout. Implementation is trivial — mask the action tokens' loss on a subset of samples.
Limitations and Open Questions
The paper acknowledges that prompt action is only a behavioral proxy for "how much action information is recoverable from the prompt," not a direct measurement of internal computation. Some judgments rely on GPT-5.4 as a judge, which may introduce evaluation bias. The paper doesn't claim CoT is useless — only that "the gains from standard CoT supervision should not be read as evidence of enhanced CoT revision ability."
A deeper question: if training doesn't improve reasoning, what would? The intervention merely makes CoT somewhat useful again; it doesn't make CoT the dominant path. Genuinely improving reasoning may require structural redesign — balancing gradient flow between CoT and action, or architecturally preventing the prompt→action shortcut.
Closing
This paper does what science should but rarely does: refute a widely assumed hypothesis. The mainstream story says "CoT training improves reasoning." Within the measured environments, the evidence doesn't support that. Training improves direct prompt→action prediction, CoT's relative advantage stays flat or shrinks, and conflicting chains increasingly fail to move actions — none of which looks like "reasoning getting stronger."
Feynman: "If you can't explain something simply, you don't understand it." This paper's variant: if the model can give equally good answers without reasoning, it probably wasn't reasoning in the first place.
Sometimes scientific progress isn't discovering a new continent — it's discovering that a continent on the map was never there.
---
Paper: *Where Do CoT Training Gains Land in LLM based Agents?* Authors: Jingyu Liu, Zhiwen Wang, Yuxin Jing, Huanyu Zhou, Yong Liu Institutions: Gaoling School of AI, Renmin University + ByteDance arXiv: 2606.26935 Code: not publicly released