Latent Thought Flow: GFlowNet-Based Latent Reasoning Beats Chain-of-Thought with +9.5% Accuracy and 27.2% Shorter Reasoning

Forum topic · 小凯 · 2026-06-18

Summary

Researchers from Singapore Management University and Ant Group propose Latent Thought Flow (LTF), a method that moves LLM reasoning from discrete token space into continuous latent space. Instead of decoding explicit chain-of-thought tokens—bottlenecked by linguistic overhead—LTF models reasoning as variable-length continuous latent trajectories trained with a GFlowNet objective. Unlike RL approaches such as GRPO, which collapse the posterior onto a few high-reward modes, GFlowNet learns a sampling policy whose terminal probability is proportional to a reward combining answer accuracy and computation cost. LTF adds entropy-weighted subtrajectory balance and a reference prior anchored on explicit reasoning chain embeddings that anneals during training. On GSM8K-Aug, ASDiv-Aug, and DU across LLaMA-3.2 1B and LLaMA-3.1 8B backbones, LTF improves accuracy by 9.5% on average while cutting reasoning length by 27.2%, and transfers well to out-of-domain tasks like GSM-HARD, SVAMP, and AQUA-RAT (+6.0% accuracy, -19.9% length). The variable-length design enables adaptive computation and test-time scaling via trajectory sampling.

Latent Thought Flow (LTF), proposed by researchers from Singapore Management University and Ant Group, breaks the "linguistic space bottleneck" of explicit chain-of-thought (CoT) reasoning by learning to reason in the latent space of LLMs using GFlowNet. Reported results: up to +9.5% accuracy and -27.2% reasoning length versus strong latent-reasoning baselines.

The Problem: Explicit CoT's Token Overhead

Explicit CoT forces every intermediate reasoning step to be decoded into discrete tokens—like a mathematician who must speak every number aloud. The paper identifies three issues:

High inference overhead (every token passes through the decoder)
Verbose, human-written reasoning chains
An inherent trade-off between accuracy and efficiency

Existing latent reasoning methods (pause tokens, CoT compression, continuous thought) mostly learn deterministic or reward-maximizing paths. The key missing piece: a distributional view—one problem may have multiple correct reasoning paths of varying cost, and no principled method allocates probability across them.

Core Idea: Learn a Distribution, Not a Path

LTF models reasoning as variable-length continuous latent trajectories and trains the sampler with a GFlowNet. GFlowNet's defining property: the probability of a terminal sample is proportional to its unnormalized reward. This differs fundamentally from RL:

| | RL (GRPO) | GFlowNet | |---|---|---| | Objective | Maximize expected reward | Match the reward-induced posterior | | Outcome | Posterior collapse to few modes | Diverse high-reward solutions preserved | | Training signal | Sparse reward | Flow-matching constraint |

Four Key Components

1. Variable-length latent trajectories: τ = (z₁, ..., z_T, ⊥), where each z_t is a continuous thought state and ⊥ denotes adaptive stopping (T can be 0 for direct answers). The Gaussian sampler enables gradient backpropagation via reparameterization.

2. Accuracy–efficiency reward: combines answer correctness and reasoning length, inducing p*(τ | x, y) ∝ R(τ)—accurate and short paths get higher probability.

3. Subtrajectory balance with entropy weighting: every prefix state may terminate and receive reward, avoiding extra flow estimators; entropy weighting encourages exploration of diverse subtrajectories.

4. Reference prior regularization: embeddings of explicit reasoning chains serve as anchors in latent space; prior strength anneals from 3.0 to 0.1 over 100 epochs.

Experimental Results

Fine-tuning: LLaMA-3.2 1B: 34.58% → 37.09% vs. ReGuLaR; LLaMA-3.1 8B: 50.14% → 53.14%. Average: +9.5% accuracy, -27.2% reasoning length; vs. CoLaR/ReGuLaR: +12.9% accuracy, -34.5% length.
Transfer: +6.0% accuracy, -19.9% length on GSM-HARD, SVAMP, MultiArith, AQUA-RAT, MATH.
Ablations: GFlowNet (DB) vs. GRPO raises accuracy 47.49% → 55.98% while shortening reasoning 12.25 → 7.28; entropy weighting gains grow with sampling diversity (S=20: +0.95%); annealed prior works best.
Test-time scaling: sampling multiple latent trajectories with voting improves accuracy without decoding long chains.

Why It Matters

Paradigm shift from token reasoning to vector reasoning—thinking in concepts rather than speaking every thought aloud.
Distributional perspective avoids posterior collapse, preserving diverse solution paths.
Adaptive computation: simple questions get T=0; harder ones get more latent steps.
Industry relevance: Ant Group's involvement signals practical demand for fast, cheap reasoning in risk control, customer service, and advisory; the LoRA-based architecture deploys on existing models without full retraining.

Limitations & Future Work

Experiments limited to text/math tasks; multimodal extension is open.
Interpretability of continuous latent states remains unresolved.
Future: multimodal latent reasoning, integration with o1-style test-time computation, lighter deployment.

References

Zou, X., Huang, J., Li, J., & Zhou, P. "Latent Thought Flow: Efficient Latent Reasoning in Large Language Models." arXiv:2606.16222 (2026).
Bengio, Y. et al. "GFlowNet Foundations." (2021–2023).
Li, Z. et al. "Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs." (ReGuLaR, 2025).
Li, J. et al. "Chain of Latent Regression: Enhancing LLM Reasoning via Latent Reasoning." (CoLaR, 2025).