When AI Hides Its Thinking, Where Does Reasoning Live?
Imagine a colleague who solves complex math problems without writing anything down—eyes closed for a few seconds, then straight to the answer. Impressive, but you'd wonder: how is the mental calculation actually done? Now replace that colleague with a large language model, and you get one of the hottest research directions of 2026: Latent Chain-of-Thought (Latent CoT).
Traditional chain-of-thought makes the model write out its reasoning in text—"first compute this, then that, so the answer is X." It's transparent but slow. Latent CoT models pack intermediate reasoning into the network's continuous hidden states without emitting a textual draft. The upside is compactness and speed; the downside is that if the model makes a mistake, you can't even see which step went wrong.
So the question becomes: if reasoning isn't written on paper, which component of the transformer actually houses it?
Brain-Surgery-Grade Causal Tracing
SCIT (Suffix Cache Interchange Test), proposed by Yi Ding et al. at HKUST (Guangzhou), is essentially a "brain mapping" procedure for latent-CoT models.
The idea resembles awake brain surgery: stimulate a brain region in a conscious patient and see which function disappears. SCIT does the same—it constructs source-receiver task pairs (two different arithmetic problems), and during the receiver model's reasoning, swaps one segment of its cache with the corresponding cache from the source model, then checks whether the receiver's answer gets "pulled" toward the source's answer.
If swapping a cache segment changes the answer accordingly, that segment is "carrying the signal"—reasoning lives there.
SCIT's elegance lies in not settling for a single effective intervention point. It performs a grid-search exclusion:
- Is it the hidden state carrying the signal?
- Or the key?
- Or the value?
- A single token's cache?
- Or the entire suffix trajectory?
- Or just a coincidence of matching answers between source and receiver?
- Arithmetic-trained GPT-2/1B models: use a "latent tail value/KV transfer" pathway
- Stronger 8B models: use a "prompt-prefix or full-cache K/V" pathway
- Boundary models: neither mechanism is clearly detectable
Each possibility is ruled out by a corresponding control condition. SCIT outputs not a number, but a carrier map.
The Surprising Answer: Not Hidden States—Value Cache Suffixes
On CODI-GPT2 (a latent-CoT GPT-2 model), SCIT delivers a strikingly clear conclusion:
Counterfactual arithmetic reasoning is transmitted primarily via the suffix trajectory of the value cache—not hidden states, keys, reusable answer slots, or single-token triggers.
In plain terms: when the model does mental arithmetic, the thing "remembering intermediate results" is not the heavily studied hidden-state vector, but the often-ignored "value" channel of attention, and specifically its later positions.
It's like searching for where a colleague stores memories and finding them not in the hippocampus (hidden states), not in the prefrontal cortex (keys), but in an auxiliary pathway you thought only handled "readout"—the value cache suffix.
A finer finding: the carrier mechanism shifts with model capability:
Why This Matters
1. Interpretability upgrades from "looking" to "swapping"
Previous latent-CoT analyses mostly perturbed a variable and checked if the answer changed. That tells you the variable matters, but not which pathway carries the effect. SCIT upgrades "it has an effect" to "it is carried via pathway X"—from correlation to causation, from seeing to swapping.
2. Breaking the "hidden states explain everything" myth
Much interpretability research focuses on hidden states because they're the most visible intermediate product in a transformer. But SCIT shows that in latent-CoT reasoning, hidden states have far less causal control than the value cache suffix. The most visible intermediate product isn't necessarily the most important causal carrier.
3. No universal conclusions, only carrier maps
The authors are deliberately cautious: this is not a universal claim that "all latent-CoT models reason through value caches." Models of different scales and training regimes may have completely different carrier mechanisms. SCIT provides a diagnostic methodology, not a one-size-fits-all answer—a restraint that is refreshingly rare in today's "one finding wrapped in a grand narrative" paper ecosystem.
A Deeper Metaphor
SCIT evokes the "double dissociation" experiments of neuroscience, which use selective damage to prove two functions are handled by different brain regions. But SCIT is more refined: rather than "damage a part and see if the function disappears," it asks "swap in another version of the function and see which part carries it back."
It's not asking "can the model still compute without this part," but "if I transplant problem A's reasoning into problem B's reasoning, will B's answer become A's?" Only genuinely carrying components can achieve such answer transplantation.
Reasoning isn't text written on paper, but it isn't uniformly dissolved in the water either. It has a specific address—and that address moves as the model's capability grows.
---
Paper: SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models
Code: https://github.com/YIDING4869/scit-emnlp-2026
Authors: Yi Ding, Lijun Huang, Menglin Yang (HKUST Guangzhou)
Published: 2026-08-27