Reusing the Same SSM Block Across Depth: Looped State-Space Models Beat Independent Parameters

Forum topic · 小凯 · 2026-05-18

Summary

A post on zhichai.net discusses the paper "Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification" by Farsang, Hasani, Rus, and Grosu (MIT CSAIL and TU Wien). Instead of stacking L state-space model (SSM) layers with independent parameters, the authors reuse a single parameter block k parameters L times along the depth dimension. Although the hypothesis space of the looped model is strictly smaller than that of the parameter-independent model, experiments across four SSM architectures (LRU, S5, LinOSS, LrcSSM) and six time-series classification benchmarks show that the looped model consistently matches or outperforms its counterpart, indicating that depth-wise parameter sharing acts as a beneficial inductive bias that simplifies optimization rather than limiting expressivity. The paper also introduces input reshaping for low- and high-dimensional inputs, yielding 1-6% accuracy gains, with improvements that stack independently with depth recurrence. The post raises open questions about choosing the optimal loop count, generalizable reshaping heuristics, and applicability beyond time series.

Overview

State-space models (SSMs) have become strong competitors to Transformers, especially on long-sequence tasks. Their core operation is recurrence along the sequence dimension. This post covers a paper that explores a less-studied dimension: recurrence along depth.

The work, *Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification* (Farsang, Hasani, Rus, Grosu — MIT CSAIL + TU Wien, arXiv:2605.16048), compares two designs:

Standard: stack L SSM layers, each with its own parameters.
Looped: use a single SSM block (k parameters) and loop it L times along depth, reusing the same parameter block.

Key Findings

Theory: the looped model's hypothesis space is strictly a subset of the parameter-independent model's, since the latter contains the former as a special case.
Practice contradicts theory: across 4 SSM architectures (LRU, S5, LinOSS, LrcSSM) and 6 time-series classification benchmarks, the looped model consistently matches or beats the parameter-independent model.
Interpretation: since the larger model strictly contains the smaller one, the advantage cannot come from expressivity. The conclusion is that parameter sharing along depth forms a beneficial inductive bias that simplifies optimization.

Input Reshaping

The paper also highlights an overlooked design axis — input reshaping:

For low-dimensional inputs, concatenate consecutive time steps to increase information density per step.
For high-dimensional inputs, flatten the joint feature–time dimension and re-chunk it.

Both strategies deliver 1–6% accuracy gains across all models. Depth recurrence and input reshaping are independent improvements, and their benefits stack when combined.

Open Questions (from the author)

1. How to choose the optimal loop count L — is it related to the temporal scale of the task? 2. Reshaping strategies (concatenation scheme, chunk size) depend on dataset properties — are there general heuristics? 3. Does depth recurrence remain effective on non-time-series tasks such as NLP?

References

1. Farsang, M., Hasani, R., Rus, D., & Grosu, R. (2026). *Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification*. arXiv:2605.16048 [cs.LG]. 2. Gu, A., & Dao, T. (2023). *Mamba: Linear-Time Sequence Modeling with Selective State Spaces*. NeurIPS. 3. Lan, Z., et al. (2020). *ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations*. ICLR.