The setup: a suspiciously good result
Imagine you design a new decoder architecture and train it on OpenWebText for 20,000 steps. Validation loss drops from the baseline's 3.5958 to 1.4259 — a reduction of over 60%. Cause for celebration?
Not quite. When you let the model generate freely, it outputs gibberish:
> self un un state un low un low low heavy self heavy un highly un heavy un self un self un heavy inf self un un highly low heavy deep heavy kind low large form form un particularly large very largely very small...
The baseline, for comparison, produces coherent text:
> too high. "Don't believe me when you get to war," he said by phone. "This is something that the government is supposed to protect," he said.
A model with 60% lower validation loss that collapses catastrophically at generation time points to a hidden, dangerous problem: future-token leakage. A paper by Huiting Liu and Jinsong Wu (Guilin University of Electronic Technology / University of Chile) shows that even with a perfectly correct causal attention mask, a model can "peek" at future tokens through auxiliary paths, manufacturing fake architectural gains.
The problem: causal masks are not enough
In a standard decoder Transformer, causal self-attention ensures position \(t\) only sees \(x_{1:t}\). But a Transformer layer contains more than attention:
- Sequence-level pooling
- Temporal convolutions
- State aggregation
- Routing decisions
- Cross-time normalization statistics
- CDR-token-fix: \(s_i = h_i\) (fully local)
- CDR-prefix-mean-fix: \(s_i = \frac{1}{i} \sum_{j=1}^{i} h_j\)
- Clean controls: GPT-2, Qwen2.5-0.5B, and Mamba-130M pass all 72 test pairs in float32 (max logit difference 0.0) — no false positives.
- Fault injection: injecting a mild (\(\alpha = 0.01\)) full-sequence summary broadcast flips all three models from 100% pass to 0% pass.
- Precision sensitivity: in float16/bfloat16, numerical drift between execution paths grows substantially (GPT-2 max difference 4.0 in bf16), so audit thresholds must be calibrated per precision and runtime path.
- Title: Beyond Causal Masks: Auditing and Localizing Hidden Future-Token Leakage in Adaptive Sequence Models
- Authors: Huiting Liu, Jinsong Wu
- Affiliations: School of Artificial Intelligence, Guilin University of Electronic Technology / Department of Electrical Engineering, University of Chile
- arXiv: 2610.31522
- Code: supplementary material includes audit and generation code, saved result files, a fixed OpenWebText prompt set, and raw continuations
Each auxiliary operation has its own information-flow path. If your causal mask only governs attention while ignoring these side paths, the model can peek at the future through a side door.
Case study: CDR-AttnRes
The core example is CDR-AttnRes (Content-Conditioned Depth Routing for Attention Residuals), a residual-routing decoder inspired by Kimi's Attention Residuals work. A dynamic routing query controls the mixture weights of residual sources from different depths, applied before attention and MLP layers.
The bug is in how the query vector is constructed:
1. Full-sequence pooling: the entire hidden state sequence \(h_{1:T}\) is pooled over time into a single vector \(s\) 2. Broadcast to all positions: \(s\) is projected into a shared query correction \(\Delta q_{shared}\) added to every token's base query
Step 1 pools over future tokens; step 2 injects that future-informed summary into every prefix position. The attention mask is correct, but position \(i\)'s routing query already contains information from \(x_{i+1:T}\). The model isn't predicting the future — it's recalling it.
The diagnostic: prefix-invariance testing
The paper's audit is elegant: if a model is truly causal, changing the suffix of a sequence must not affect outputs at prefix positions.
Procedure: 1. Construct pairs \((x, x')\) sharing prefix \(x_{1:t}\) but differing in the suffix 2. Forward both through the model 3. Compare prefix outputs \(f(x)_{1:t}\) vs \(f(x')_{1:t}\) 4. Differences beyond numerical tolerance (\(\epsilon = 10^{-5}\)) indicate leakage
The paper quantifies leakage with LeakScore:
Auxiliary metrics include top-1 flip rate, affected-token rate, and JS divergence between softmax distributions.
Audit results: damning
| Metric | Baseline | CDR-last 24L (leaky) | |--------|----------|----------------------| | Validation loss | 3.5958 | 1.4259 | | LeakScore | 0.0 | 34.22 | | Top-1 flip rate | 0.0% | 95.1% | | Reference perplexity (generation) | 16.8 | 700.5 | | Audit verdict | pass | fail |
95.1% of prefix positions flip their top-scored token when the suffix changes — predictions at prefix positions depend almost entirely on leaked future information. FirstLeakLayer localization shows leakage begins at layer 0, since the shared query correction is injected before the backbone.
The fix erases all "gains"
Two causal repairs keep the low-rank correction structure but make \(s_i\) causal:
| Variant | Val loss | LeakScore | Ref. ppl | Audit | |---------|----------|-----------|----------|-------| | CDR-token-fix | 3.6766 | 0.0 | 15.3 | pass | | CDR-prefix-mean-fix | 3.6731 | 0.0 | 18.1 | pass | | Baseline | 3.6900 | 0.0 | 19.8 | pass | | CDR-last (leaky) | 2.5488 | 11.31 | 695.5 | fail |
After the fix, validation loss returns to baseline level (~3.67 vs 3.69) and perplexity normalizes. The "breakthrough" 1.43 loss was entirely an artifact of future-token leakage.
Systematic audit of six auxiliary module families
The paper constructs six synthetic non-causal/causal pairs:
| Module | Non-causal version | LeakScore | |--------|--------------------|-----------| | Pool mean | noncausal_pool_mean | 0.076 | | Pool last | noncausal_pool_last | 0.510 | | Same-padded conv1d | noncausal_samepad_conv1d | 0.617 | | Full-KV linear attention | noncausal_fullkv_linear_attn | 0.127 | | Global routing | noncausal_global_route | 0.003 | | Global norm | noncausal_global_norm | 0.032 |
All six non-causal versions fail (0/3 pass); all causal versions pass (3/3). Even global routing's tiny LeakScore of 0.003 is far above the \(10^{-5}\) tolerance — leakage is not about magnitude, only presence.
Auditing real models
Key takeaways
1. Validation loss is a scalar illusion. It cannot distinguish "learning useful prediction" from "peeking at the future." A 3.60 → 1.43 drop can be pure leakage. 2. Causal mask ≠ causal model. Causality is a property that must hold on *every* information path in the computation graph — attention, pooling, routing, normalization statistics, state updates, temporal convolutions. 3. The teacher-forcing trap. Under teacher forcing, leaked future info just looks like extra correct context. In free generation, that future disappears — hence low validation loss plus terrible generation is the classic leakage signature. 4. Fix the structure, not the content. The repairs keep pooling and routing but change their information-flow direction (prefix-only instead of full-sequence).
Practical audit checklist
1. Prefix-invariance test: 72 sequence pairs sharing prefixes, tolerance \(\epsilon = 10^{-5}\) (float32) 2. LeakScore computation: report top-1 flip rate and affected-token rate alongside 3. FirstLeakLayer localization: per-layer hidden-state leak to find the entry point 4. Implementation review: inspect all auxiliary ops near the first leak layer (pooling, routing, normalization, state aggregation) 5. Fix verification: replace suspect modules with causal versions, retrain — if the "gain" vanishes, it came from leakage 6. Free-generation evaluation: don't trust validation loss alone; check generation quality with reference-model perplexity