MUX: Continuous Reasoning via Multiplexed Tokens — Compressing LLM Chain-of-Thought into Lossless Latent Tokens

Forum topic · 小凯 · 2026-07-22

Summary

This zhichai.net forum post explains MUX (Continuous Reasoning via Multiplexed Tokens, arXiv:2607.18264), a technique that compresses discrete chain-of-thought (CoT) reasoning in large language models into continuous latent tokens. The author argues that standard CoT is inefficient: each generated token requires a full forward pass, and many tokens merely express thoughts ('first', 'so we have...') rather than perform computation — a costly bottleneck as reasoning models like OpenAI o1/o3 and DeepSeek-R1 generate tens of thousands of tokens per query. MUX borrows multiplexing from telecommunications: one latent token linearly superposes multiple reasoning subwords using position-dependent geometric-decay weights, a scheme the paper proves is lossless and invertible, avoiding 'latent collapse' that plagues prior latent-reasoning methods. Evaluated across 32 settings and four models, MUX reportedly outperforms strong latent-reasoning baselines on math, logic, code, commonsense, and reading tasks. The post also covers demultiplexing (recovering subwords from latent tokens), probing analyses showing latent tokens encode faithful, interpretable reasoning, parallel exploration of multiple hypotheses in continuous latent space, ablation studies (lossless vs lossy compression, geometric vs uniform weights), limitations (training complexity, hyperparameter sensitivity, task scope), and philosophical implications for non-linguistic 'mentalese'-style thinking.

MUX: Compressing AI Reasoning with Multiplexed Latent Tokens

This post from zhichai.net is an in-depth explainer of the paper MUX: Continuous Reasoning via Multiplexed Tokens (Suleymanzade, Gozeten, Bronstein, Ceylan & Kim, arXiv:2607.18264). Below is a structured English rendering of the original Chinese article.

The Problem: Chain-of-Thought Is Verbally Wasteful

The article opens with an analogy: a genius at a dinner party who insists on narrating every step of a mental calculation (23 × 47) in full sentences, taking minutes while others finish silently. Current LLMs face the same issue.

Chain-of-Thought (CoT) improves accuracy (e.g., from ~40% to 70%+ on math) but at a large efficiency cost.
Every reasoning step generates one token, each requiring a full forward pass. As the paper puts it: *"Each reasoning step conveys only a single subword, and many are spent expressing a thought instead of carrying out computation."*
Filler tokens ("let me think...", "so we have...") contribute nothing to the actual computation.
With reasoning models (OpenAI o1/o3, DeepSeek-R1) generating tens of thousands of CoT tokens, inference cost — often 70%+ of total deployment cost — is dominated by token count.

Existing mitigations fall short: 1. Shorter CoT — drops intermediate steps, raising error rates. 2. External tools (calculators, Python) — adds complexity; tool calls also cost tokens. 3. Latent reasoning — promising, but prior methods suffer from latent collapse (models learn shortcuts rather than genuine reasoning). MUX improves on this third approach.

How MUX Works: Lossless Multiplexing

MUX trains a model to compress discrete natural-language reasoning steps into continuous latent tokens, where each latent token is a linear superposition of multiple subwords:

\[latent = a_1 w_1 + a_2 w_2 + a_3 w_3 + a_4 w_4 + a_5 w_5\]

Borrowing multiplexing from telecommunications (many signals over one channel), MUX uses position-dependent geometric-decay weights (weight_i = r^i, with 0 < r < 1):

The paper proves this multiplexing is lossless: the original subwords can be fully recovered via demultiplexing, because the weight matrix is invertible.
Losslessness is the key safeguard against latent collapse — it forces the model to learn meaningful compression instead of shortcuts, analogous to "compression is understanding" from information theory.
Unlike discrete step-by-step CoT, reasoning flows through a smooth continuous latent space.

Results and Capabilities

Benchmarks: Evaluated across 32 settings and four models, spanning math (GSM8K etc.), logic, code generation, commonsense, and reading comprehension. MUX outperformed strong latent-reasoning baselines in all settings.
Efficiency: While the paper gives no exact compression ratio, the author estimates latent tokens could reduce CoT token counts by roughly an order of magnitude, implying 5–10× speedups and 80–90% cost/latency reductions (author's estimate, not a paper claim).
Parallel exploration: The continuous latent space lets the model consider multiple hypotheses simultaneously rather than sequentially — hinting at "divergent" machine thinking.
Interpretability: Probing analyses show latent tokens encode faithful, interpretable reasoning; decoding them back to natural language reveals the model's genuine intermediate reasoning.
Ablations: Lossless multiplexing beats lossy compression (which causes collapse); geometric decay beats uniform weights; large latent dimensions are unnecessary.

Broader Implications

Cognitive science angle: MUX supports the view that effective reasoning need not occur in natural language — echoing the "mentalese" hypothesis over linguistic determinism.
Practical outlook: cheaper API calls, lower latency, and potentially running reasoning-grade models on edge devices — though the post notes the edge-device claim is speculative extrapolation.
Limitations acknowledged: extra training complexity for multiplexing/demultiplexing, untested applicability to creative or affective tasks, decode-step error risk, sensitivity to the decay parameter r, and required architectural changes limiting compatibility with existing models.

References Cited in the Post

MUX paper: arXiv:2607.18264
Wei et al. (2022), Chain-of-Thought Prompting, NeurIPS 2022
Havrilla et al. (2024), arXiv:2403.04642
Deng et al. (2023), arXiv:2312.10997
Kahneman (2011), *Thinking, Fast and Slow*
Shannon (1948), *A Mathematical Theory of Communication*
OpenAI (2024), Learning to Reason with LLMs; DeepSeek-R1, arXiv:2501.12948