SCIT: Locating Where Latent Chain-of-Thought Reasoning Lives in a Transformer

Forum topic · 小凯 · 2026-08-30

Summary

SCIT (Suffix Cache Interchange Test), proposed by Yi Ding and colleagues at HKUST (Guangzhou), is a causal intervention method for localizing where reasoning is carried inside latent chain-of-thought (Latent CoT) language models. Inspired by brain-mapping experiments, SCIT builds source-receiver task pairs and swaps segments of cache (hidden states, keys, values, single tokens, or full suffix trajectories) between two models solving different arithmetic problems, checking whether the receiver's answer is steered toward the source's answer. On CODI-GPT2, counterfactual arithmetic reasoning is carried mainly by the suffix trajectory of the value cache—not by hidden states, keys, reusable answer slots, or single-token triggers. Across model scales, SCIT also identifies a carrier-regime shift: arithmetic-capable 1B models rely on latent tail value/KV transfer, stronger 8B models on prompt-prefix or full-cache K/V, while borderline models show neither clearly. SCIT upgrades interpretability from correlational perturbation to causal, pathway-level localization, challenging the assumption that hidden states are the primary causal carriers. Paper: https://arxiv.org/abs/2608.27265; code: https://github.com/YIDING4869/scit-emnlp-2026.

When AI Hides Its Thinking, Where Does the Reasoning Actually Live?

Imagine a colleague who, when asked a complex math problem, never writes out steps—just thinks for a few seconds and states the answer. You'd admire them, but you'd also wonder: how does their mental calculation actually work?

Replace the colleague with a large language model, and you get one of the hottest research directions: Latent Chain-of-Thought (Latent CoT).

Traditional chain-of-thought makes the model write out its reasoning in text. That's transparent but slow. Latent CoT models pack intermediate reasoning into the network's continuous hidden states instead of emitting textual drafts. The result is compact and fast—but if the model gets something wrong, you can't even see where it failed.

So the question becomes: if reasoning isn't written on paper, which part of the transformer does it live in?

Brain-Surgery-Grade Causal Tracing

SCIT (Suffix Cache Interchange Test), proposed by Yi Ding et al. at HKUST (Guangzhou), is essentially a "brain localization" procedure for latent-CoT models.

The idea resembles awake brain surgery: stimulate a region and see which function disappears. SCIT constructs a pair of source-receiver tasks (two different arithmetic problems). During the receiver model's reasoning, it swaps a segment of the receiver's cache with the corresponding cache from the source model, then checks whether the receiver's answer gets pulled toward the source's answer.

If swapping a given cache block changes the answer, that block is "carrying" the reasoning.

Crucially, SCIT doesn't stop at finding one effective intervention point. It performs a grid-search-style elimination:

Is it the hidden state carrying the signal?
Or the key?
Or the value?
A single token's cache?
The full suffix trajectory?
Or just coincidence—source and receiver happened to share an answer?

Each possibility is ruled out by a matched control. SCIT outputs not a single number, but a carrier map.

The Surprising Answer: Not Hidden States, But the Value-Cache Suffix

On CODI-GPT2 (a latent-CoT GPT-2 model), SCIT delivers a crisp conclusion:

Counterfactual arithmetic reasoning is carried primarily by the suffix trajectory of the value cache—not by hidden states, keys, reusable answer slots, or single-token triggers.

In plain terms: when the model does mental math, the thing "remembering intermediate results" isn't the heavily-studied hidden-state vector, but the often-overlooked value channel of the attention mechanism—and specifically its later positions.

A finer finding is that the carrier mechanism shifts with model capability:

Arithmetic-capable GPT-2/1B models: a "latent tail value/KV transfer" pathway
Stronger 8B models: a "prompt-prefix or full-cache K/V" pathway
Borderline models: neither mechanism clearly present

SCIT calls this a carrier-regime shift—the stronger the model, the different the "address" of its reasoning.

Why This Matters

1. Interpretability upgrades from "looking" to "swapping." Prior latent-CoT analyses mostly perturb a variable and see whether the answer changes—showing influence, but not the pathway. SCIT upgrades "it matters" to "it is carried via pathway X": from correlation to causation.

2. Breaking the "hidden states do everything" myth. Much interpretability work focuses on hidden states because they're the most visible intermediate product. But SCIT finds they are far less causally controlling than the value-cache suffix. The most visible intermediate product isn't necessarily the most important causal carrier.

3. No universal conclusion, only carrier maps. The authors are notably restrained: this is not a claim that "all latent-CoT models reason through value caches." Models of different scales and training regimes may carry reasoning completely differently. SCIT offers a diagnostic method, not a universal answer—a restraint that's refreshing in an era of one-finding-fits-all-grand-narrative papers.

A Deeper Metaphor

SCIT recalls double-dissociation experiments in neuroscience, but goes further: instead of "lesion a part and see if function vanishes," it asks "swap the function with another version and see which part can bring it back." Only a genuine carrier can pull off this kind of "answer transplantation."

Reasoning isn't text written on paper, but it isn't uniformly dissolved in the water either. It has a specific address—and that address moves as the model's capability grows.

---

Paper: SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models

Code: https://github.com/YIDING4869/scit-emnlp-2026

Authors: Yi Ding, Lijun Huang, Menglin Yang (HKUST Guangzhou)

Published: 2026-08-27