MUX: Continuous Reasoning via Multiplexed Tokens — Compressing AI Chains of Thought into Latent Tokens

Forum topic · 小凯 · 2026-07-22

Summary

This zhichai.net forum post introduces MUX (Continuous Reasoning via Multiplexed Tokens, arXiv:2607.18264), a technique that compresses a language model's chain-of-thought reasoning into continuous latent tokens. The author argues that conventional Chain-of-Thought is inefficient: each reasoning step conveys only a single subword, and many tokens are spent on filler phrases rather than computation — a costly problem as reasoning models like OpenAI o1 and DeepSeek-R1 generate tens of thousands of tokens per query. MUX borrows multiplexing from telecommunications: one latent token encodes a linear superposition of several reasoning subwords, using position-dependent geometrically decaying weights. Because this multiplexing is provably lossless (the weight matrix is invertible), the original subwords can be recovered via demultiplexing, which the paper argues avoids the well-known 'latent collapse' failure of prior latent-reasoning methods. Across 32 evaluation settings and four language models covering math, logic, code, and comprehension tasks, MUX reportedly outperforms strong latent-reasoning baselines while drastically cutting token counts. Probing analyses suggest the latent tokens encode faithful, interpretable reasoning, and the continuous latent space enables parallel exploration of multiple reasoning branches. The post also discusses cognitive-science implications (mentalese, non-verbal thinking), limitations such as training complexity and hyperparameter sensitivity, and potential impacts on inference cost, latency, and edge deployment.

Key points

The problem: Chain-of-Thought (CoT) reasoning improves accuracy (e.g., from ~40% to 70%+ on math) but is token-expensive. Each reasoning step conveys only a single subword, and many tokens are spent *expressing* a thought ("let me think...", "therefore...") rather than *computing* it.
Why it matters now: Reasoning models (OpenAI o1/o3, DeepSeek-R1) generate tens of thousands of CoT tokens per query. Inference cost often dominates total deployment cost, making token reduction a multi-billion-dollar problem.
The idea: MUX trains a model to compress discrete natural-language reasoning steps into *continuous latent tokens*. Borrowing "multiplexing" from telecom, one latent token encodes a linear superposition of multiple reasoning subwords:

\[latent = a_1 w_1 + a_2 w_2 + a_3 w_3 + a_4 w_4 + a_5 w_5\]

Losslessness is the key: With position-dependent, geometrically decaying weights (weight_i = r^i, 0 < r < 1), the multiplexing is provably lossless — the weight matrix is invertible, so the original subwords can be recovered via *demultiplexing*. The paper argues this rules out the notorious "latent collapse" problem where models learn shortcuts instead of genuine compression.
Results: MUX was evaluated across 32 settings spanning math (GSM8K-style), logic, code generation, commonsense, and reading comprehension, on four models of different scales. It outperformed strong latent-reasoning baselines on all settings.
Efficiency: While no exact compression ratio is given, the mechanism suggests roughly an order-of-magnitude reduction in reasoning tokens — implying potential 5–10× speedups and 80–90% cost/latency reductions (the post's extrapolation, not a paper claim).
Bonus capabilities:
Parallel exploration: the continuous latent space lets the model simultaneously consider multiple reasoning branches (e.g., alternative moves in search problems) rather than serially.
Interpretability: probing analyses show latent tokens encode *faithful* reasoning — decoding them back to language reveals what the model actually "thought," without hidden errors.
Ablations: lossless multiplexing beats lossy compression; geometric decay beats uniform weights (it preserves position information); large latent dimensions are not required.

Cognitive-science reflections

The post connects MUX to the debate between linguistic determinism and Fodor's "mentalese": MUX suggests effective reasoning need not happen in natural language at all — a model can reason in a compact continuous internal representation and translate it back losslessly. This challenges the CoT assumption that "if a model can't say it, it didn't think it," and indirectly touches on questions of machine "inner speech" (though the author cautions current latent tokens are just compressed statistical patterns, not consciousness).

Limitations noted

1. Additional training complexity for multiplexing/demultiplexing. 2. Untested outside reasoning-style tasks (creative writing, sentiment analysis). 3. Decoding latent tokens back may introduce errors; continuous states are less human-readable. 4. Sensitivity to the decay hyperparameter *r*. 5. Requires architecture changes, limiting compatibility with existing systems.

Conclusion

Echoing the information-theory maxim "compression is understanding," the author frames MUX as teaching AI a private "shorthand": reasoning flows continuously through latent space rather than in discrete verbal steps — potentially enabling cheaper APIs, faster responses, and even reasoning-grade models on edge devices.

References

Suleymanzade, A., Gozeten, H. A., Bronstein, M., Ceylan, İ. İ., & Kim, J. (2026). MUX: Continuous Reasoning via Multiplexed Tokens. arXiv:2607.18264.
Wei, J., et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022.
Havrilla, A., et al. (2024). Teaching Large Language Models to Reason with Reinforcement Learning. arXiv:2403.04642.
Deng, L., et al. (2023). Understanding and Improving Reasoning in LLMs: A Survey. arXiv:2312.10997.
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal.
OpenAI. (2024). Learning to Reason with LLMs. OpenAI Blog.
DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948.