Temporal Credit Is Free: Why Did It Take 30 Years to Realize the Jacobian Is Unnecessary in Online RNN Learning?

Forum topic · 小凯 · 2026-04-01

Summary

This post is a detailed Chinese-language analysis of the paper 'Temporal Credit Is Free' (arXiv:2603.28750), which challenges the 35-year-old assumption behind Real-Time Recurrent Learning (RTRL). The paper argues that the state-Jacobian term in RTRL and its approximations (UORO, KF-RTRL, Sparse RTRL) is largely unnecessary: the hidden state itself already carries temporal credit through the forward pass. The author identifies two overlooked errors in prior online learning methods: eligibility-trace decay values (e.g., 0.95) are massively miscalibrated relative to measured self-propagation factors (~0.01/step), and a 100x gradient-norm imbalance between recurrent and output weights cripples SGD. With two simple fixes—trace decay set to zero and Adam/RMSprop's second-moment (beta_2) normalization—immediate derivatives alone match or exceed full RTRL at O(n^2) memory instead of O(n^3). Experiments span 10 architectures, cross-session BCI decoding, streaming ML benchmarks, and LoRA-based adaptation of models up to 7B parameters. A practical architecture rule emerges: per-parameter gradient normalization is required when nonlinear state updates compress gradients and no output bypass exists.

Temporal Credit Is Free: Why Did It Take 30 Years to Realize the Jacobian Is Unnecessary?

*English summary of a Chinese forum deep-dive (published April 2026) based on arXiv:2603.28750.*

Key points

The core claim of the paper: the state-Jacobian term in RTRL carries almost no gradient information beyond what the immediate derivatives already provide. The hidden state itself is the carrier of temporal credit.
Two long-standing errors explain why the field missed this: (1) eligibility-trace decay is severely miscalibrated, and (2) gradient scales across parameter groups are wildly imbalanced, making SGD unusable online.

Background: the online learning dilemma

In online learning, a model updates itself while processing a data stream. For recurrent neural networks, this raises the temporal credit assignment problem: when the model errs now, which past inputs/decisions are to blame?

The standard answer since 1989 is RTRL (Williams & Zipser), which maintains a sensitivity tensor P_t = ∂h_t/∂θ via the recursion P_t = J_t · P_{t-1} + direct gradients. The J_t · P_{t-1} term costs O(n⁴) per step, making full RTRL impractical for n > ~64. A 30-year line of approximations (UORO at O(n²), KF-RTRL at O(n³), Sparse RTRL at O(n²), LRUs at O(n)) all share the assumption that the Jacobian term carries irreplaceable gradient information. This paper argues that assumption is wrong.

Two overlooked errors

Error #1: trace decay miscalibration

Eligibility traces use e_t = λ·e_{t-1} + direct gradient with λ typically 0.95. But in a trained vanilla RNN (n=64), the measured neuronal self-propagation factor is ≈ 0.01 per step. At λ=0.95 vs. reality 0.01, each step over-weights the past by 85x, compounding exponentially: at t=500 the accumulated trace is 6.7x larger than the immediate derivative, pointing in a different direction (cosine similarity 0.577). Traces deliver stale gradient information at the wrong scale, not useful temporal credit.

Error #2: hidden gradient-scale catastrophe

In a trained n=64 RNN, recurrent-weight gradients (∂L/∂W_hh ≈ 0.001) are 100x smaller than output-weight gradients (∂L/∂W_out ≈ 0.1), because the tanh derivative compresses gradients through the recurrence while the output path is a direct linear projection. SGD applies the same learning rate to both, so the network can adjust its output mapping but not its dynamics.

The fix: two simple corrections

1. Set trace decay = 0. Empirically, decay from 0.001 to 0.5 recovers 102–200% of full RTRL performance; decay = 0.95 (the standard default) recovers 0% on every seed. Below 0.5 is a safe plateau; 0.7 is high-variance (77±56%); 0.95 is off a cliff. 2. Use Adam/RMSprop (β₂ second-moment normalization). With immediate derivatives only:

| Optimizer | Sine task recovery | Delayed task recovery | |---|---|---| | SGD | -34% | -58% | | SGD + Momentum | 12% | -59% | | Adam β₁-only | -34% | -58% | | RMSprop (α=0.99) | 91% | 179% | | Adam (full) | 102% | 147% |

Every configuration with second-moment normalization adapts; none without it does. β₁/momentum alone adds nothing. β₂ equalizes effective learning rates across parameter groups.

Architecture rule: when is β₂ needed?

Across 10 architectures, the pattern holds: per-parameter gradient normalization is required when (1) the recurrent state update involves nonlinear/multiplicative transformations that compress gradients, AND (2) no sufficiently capacious parallel output path exists.

Need β₂: Vanilla RNN (100× gradient ratio), LSTM, CTRNN (278×), xLSTM, RWKV (∞), RetNet.
SGD suffices: Vanilla GRU (update-gate bypass, 5–19×), SSM/S4 (C/D output bypass, <6×), Block-GRU+MLP (MLP output; on Lorenz, SGD beats Adam by 50×).

Scalability and validation

| Method | Complexity | Memory at n=1024 | |---|---|---| | Immediate derivatives (decay=0) | O(n²) | 12.6 MB | | Sparse RTRL (k=4) | O(n²) | ~500 MB | | Full RTRL | O(n³) | 12.9 GB |

Real-world results:

Cross-session BCI decoding (primate reaching, 7 months of electrode drift): RMSprop α=0.99 recovers 106% of initial performance; Adam 82%; Sparse RTRL 85%; Full RTRL only 50% (2 of 5 seeds partially diverged).
Streaming ML (River Hyperplane): RNN + decay 0 + Adam reaches 0.925 accuracy vs. 0.842 for Adaptive Random Forest (+8.3 pp).
Pretrained LMs with LoRA (no replay, single pass): Adam β₂ adapts while SGD achieves near-zero adaptation everywhere — GPT-2 124M: 21.6% vs 0.0%; Mamba-130M: 51.6% vs 0.1%; TinyLlama 1.1B: 31.6% vs 0.1%; Mamba-1.4B: 133.5% vs 2.4%; Qwen2 7B: 31.3% vs 0.0%. With rank-16 adapters and 40K tokens, GPT-2 reaches 136.8%.

Why did this take 30 years?

1. Nobody measured the actual self-propagation factor. Murray's RFLO used λ=0.9, e-prop used membrane time constants, 0.95 became convention — despite the diagnostic being trivial to compute. 2. Adam didn't exist until 2014. For 25 years everyone used SGD, for which the "temporal propagation is needed" conclusion is actually true. The field never re-examined it after per-parameter normalization became standard. 3. The errors reinforced each other: believing propagation was necessary, no one investigated why immediate-only+SGD failed — and its failure seemed to confirm the belief. Nobody tried the combination immediate-only + Adam.

Deeper insight: parameter-Jacobian isotropy

In trained RNNs, the parameter Jacobian is nearly isotropic — condition numbers of 2.6–6.5 at n=64, scaling as n^0.216 — a property inherited from Gaussian initialization and preserved by training. Because gradient information is spread roughly uniformly across parameter directions, any single direction (including the immediate derivative) captures most of the relevant signal. There is no privileged subspace that temporal propagation would reveal.

Limitations

Single-layer networks in from-scratch experiments; cross-layer Jacobians in two-layer RNNs are highly anisotropic (condition number 3773) — deep recurrent networks remain open.
Tested temporal dependencies are ≤50 steps; the claim concerns gradient computation, not long-horizon credit assignment when the hidden state itself forgets.
Direct online updates of pretrained base models fail regardless of optimizer; LoRA-style adapters are required.

Practical recommendations

For online training of single-layer recurrent architectures: set trace decay to zero and use Adam or RMSprop. This matches or exceeds full RTRL at a fraction of the cost, with no Jacobian computation.
For pretrained models: use LoRA + Adam for online domain adaptation; SGD cannot.
Checklist: nonlinear state update + no output bypass → β₂ normalization is mandatory; output bypass present → SGD may suffice or be better.

Closing thought

The title is a pun: temporal credit assignment needs no expensive Jacobian propagation — the forward pass provides it for free — and we missed this for 30 years because of two simple errors that pushed everyone toward complex solutions. Sometimes scientific progress comes not from discovering new complexity, but from noticing a simple truth we were ignoring.

References

Merin, S. A. (2026). *Temporal Credit Is Free*. arXiv:2603.28750.
Williams, R. J., & Zipser, D. (1989). *A learning algorithm for continually running fully recurrent neural networks*.
Murray, J. M. (2019). *Local online learning in recurrent networks with random feedback*.
Kingma, D. P., & Ba, J. (2015). *Adam: A method for stochastic optimization*.

No official code release; the core algorithm (immediate derivatives + Adam/RMSprop) is directly implementable in any deep learning framework.