> TL;DR: Explicit CoT reasoning hits a "linguistic space bottleneck." LTF, from Singapore Management University and Ant Group, uses GFlowNet to learn reasoning directly in the latent space of LLMs — no token decoding of intermediate steps — improving accuracy by 9.5% while cutting reasoning length by 27.2%. It's not about thinking less, but thinking smarter.
---
1. The Problem: AI's "Self-Talk" Slows Down Its Thinking
The "Linguistic Space Bottleneck" of Explicit CoT
The most common approach to LLM reasoning is Chain-of-Thought (CoT) — having the model write out its intermediate thinking as text, e.g.:
> "First, I need to compute the area of the circle. The formula is πr². The radius is 5, so the area is 25π..."
This is intuitive but has a fatal flaw: every intermediate step must be decoded into discrete tokens. It's like forcing a mathematician to read every number aloud — not because they can't compute, but because the "reading aloud" itself consumes enormous resources.
The paper calls this the "linguistic space bottleneck":
- High inference overhead (every token passes through the decoder)
- Verbose reasoning chains (human-written reasoning is often wordy)
- An inherent tension between accuracy and efficiency
- Pause tokens: hidden "pause" tokens added before output for extra computation
- CoT compression: compressing long reasoning chains into short token sequences
- Continuous thought: continuous vectors replacing discrete thought tokens
- Learn deterministic paths — same input, same path
- Or follow reward-maximizing paths — only the single highest-reward route
- Lack a principled method for assigning probability across trajectories with varying correctness and cost
- Answer quality: how well the final answer matches the correct one
- Computational cost: reasoning length (shorter is better)
- Embeddings of explicit reasoning chains from training data serve as "anchors"
- Prior strength linearly decays during training (from 3.0 to 0.1 over 100 epochs)
- Early training: strong constraint keeps exploration semantically meaningful
- Late training: relaxed constraint lets the reward-driven objective dominate
- LLaMA-3.2 1B: vs. the strongest baseline ReGuLaR, accuracy 34.58% → 37.09% (+2.51%), reasoning length 3.69 → 3.34 (−9.5%)
- LLaMA-3.1 8B: accuracy 50.14% → 53.14% (+3.00%), reasoning length 3.93 → 3.37 (−14.2%)
- On average: +9.5% accuracy, −27.2% reasoning length
- Against stronger baselines CoLaR and ReGuLaR: +12.9% accuracy, −34.5% reasoning length
- On out-of-domain tasks (GSM-Hard, SVAMP, MultiArith, AQUA-RAT, MATH): +6.0% accuracy, −19.9% reasoning length
- Under extreme compression, LTF consistently beats all baselines
- +2.72% over ReGuLaR on MATH, +3.61% on AQUA-RAT
- Entropy weighting consistently improves accuracy with nearly unchanged reasoning length
- Gains grow with sampling diversity (+0.40% at S=5, +0.95% at S=20)
- GFlowNet objective (DB) vs. RL objective (GRPO): DB raises accuracy from 47.49% to 55.98% while cutting reasoning length from 12.25 to 7.28
- LTF achieves the best balance: 59.68% accuracy with reasoning length of only 1.91
- The prior is crucial early in training, preventing latent exploration from drifting
- Annealing (strong then weak) works best
- Explicit CoT ≈ forcing a person to write every thought as a sentence
- Latent reasoning ≈ letting a person think in concepts, images, and intuition, only stating the conclusion at the end
- Easy: T=0, answer directly
- Medium: 2–3 latent steps
- Hard: more steps until the model "feels" done
- Real demand for latent reasoning: inference speed and cost are critical in risk control, customer service, and robo-advisory
- A path from research to product: LTF's architecture (LoRA modules + latent reasoning head) can be deployed efficiently on existing models without retraining from scratch
- Zou, X., Huang, J., Li, J., & Zhou, P. "Latent Thought Flow: Efficient Latent Reasoning in Large Language Models." arXiv:2606.16222 (2026).
- Bengio, Y. et al. "GFlowNet Foundations." (2021–2023).
- Yang, C. et al. "Large Language Models for Automated Open-ended Scientific Discovery." (2024).
- Li, Z. et al. "Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs." (ReGuLaR, 2025).
- Li, J. et al. "Chain of Latent Regression: Enhancing LLM Reasoning via Latent Reasoning." (CoLaR, 2025).
Limitations of Existing Latent Reasoning Methods
Researchers have explored latent reasoning — thinking in continuous latent space rather than discrete token space:
But these approaches:
Key gap: no "distributional view" — for the same problem there may be multiple correct reasoning paths, some short, some long, some efficient, some redundant. Existing methods cannot assign probabilities across them.
---
2. LTF's Core Idea: "Flowing" Thought in Latent Space
2.1 Not "One Optimal Path" but "A Distribution"
LTF's core breakthrough: it models reasoning as variable-length continuous latent trajectories and trains the sampler with GFlowNet (Generative Flow Network).
What is GFlowNet?
GFlowNet learns a stochastic generation policy with a key property: > The probability of terminal samples is proportional to the unnormalized reward.
In short: instead of finding the single highest-reward path, it learns a sampler whose probability of drawing each path is proportional to that path's reward. High-reward paths are sampled more often — but not exclusively — preserving multiple good paths.
The contrast with reinforcement learning is crucial:
| | RL (GRPO) | GFlowNet | |---|---|---| | Objective | Maximize expected reward | Match reward-induced posterior distribution | | Outcome | Posterior collapses to few high-reward modes | Preserves diverse high-reward solutions | | Exploration | Prone to local optima | Naturally supports diverse reasoning | | Training signal | Sparse reward | Flow-balance constraint |
RL suffers posterior collapse: probability mass concentrates on a few high-reward modes, discarding other equally good paths. GFlowNet preserves the full distribution — essential for reasoning, since one problem often has many correct solutions.
2.2 Four Core Components of LTF
Component 1: Variable-length latent thought trajectories
Given input x, LTF samples a trajectory:
where each z_t is a "thought state" in continuous latent space and ⊥ denotes adaptive stopping. T can be 0 (direct answer, no reasoning) up to a maximum budget \(T_max\).
The sampler \(q_φ\) is Gaussian:
Via the reparameterization trick: \(z_{t+1} = μ_φ(s_t) + σ_φ(s_t) ⊙ ε\), with ε ~ N(0, I), gradients backpropagate directly.
Component 2: Accuracy-efficiency reward
A utility function \(R_{x,y}(τ)\) derived from the target answer combines:
This induces a posterior:
i.e., accurate and concise reasoning paths should have higher probability.
Component 3: Continuous Subtrajectory Balance
GFlowNet's flow-balance condition on subtrajectories:
where F(s) is the flow of state s, \(P_F\) the forward and \(P_B\) the backward transition.
LTF's innovation is allowing every prefix state to terminate: stop at any intermediate state \(s_t\), decode an answer, collect a reward. This lets flows for every prefix be computed analytically from immediate stopping rewards and stopping probabilities, avoiding extra flow estimators.
Standard Subtrajectory Balance uniformly weights all subtrajectories; LTF adds entropy weighting — higher weight for higher-entropy (more random, more diverse) subtrajectories, encouraging exploration.
Component 4: Reference-prior regularization
Unconstrained exploration in continuous latent space may "drift" away from meaningful states. LTF introduces a reference prior:
---
3. Experiments: +9.5% Accuracy, −27.2% Reasoning Length
3.1 Main Results
On GSM8K-Aug (math reasoning), ASDiv-Aug (arithmetic), and DU (math word problems), across multiple LLM backbones:
Fine-tuning setting:
Transfer learning:
Extreme compression:
3.2 Ablations
Entropy weighting and sampling diversity:
Exploration objective comparison:
Reference prior:
3.3 Test-time Scaling
LTF supports sampling multiple latent trajectories at inference and selecting or voting among them to boost accuracy — without decoding long explicit chains. This enables effective scaling with increased compute budgets.
---
4. Why LTF Matters
4.1 From "token reasoning" to "vector reasoning": a paradigm shift
LTF represents an important direction: letting AI "think" in latent space rather than "talk to itself" in token space.
It resembles human thinking:
4.2 GFlowNet's "distributional view" solves posterior collapse
RL methods like GRPO tend to teach models only a few "templates." GFlowNet's distribution-matching property preserves diverse solution paths — especially valuable for tasks requiring creative reasoning.
4.3 Adaptive computation: think more on hard problems, less on easy ones
LTF's variable length lets the model dynamically allocate compute by difficulty:
More efficient than fixed-length chains, whether explicit or latent.
4.4 What does Ant Group's involvement mean?
Authors come from Singapore Management University and Ant Group. This is not a purely academic project — as one of the world's largest fintech companies, Ant's participation signals:
---
5. Limitations and Future Directions
Limitations: 1. Experiments focus on text tasks (math reasoning); vision, speech, and multimodal settings remain unexplored 2. Theoretical analysis of generalization needs deeper work 3. Interpretability of latent reasoning is open — humans cannot understand what the continuous vectors represent
Future directions: 1. Multimodal latent reasoning (unified latent space for vision + language + reasoning) 2. Combining with test-time compute scaling (o1-style reasoning) 3. Lighter deployment schemes (currently requires training an extra latent head)
---
6. One-Sentence Summary
> LTF uses GFlowNet to learn "flowing thought" in the latent space of LLMs — no reading aloud, no self-repetition, no rambling. Higher accuracy, shorter reasoning, more diverse paths. A key step from "token reasoning" to "vector reasoning."
---