Unsilencing Latent Reasoning in Multimodal LLMs: A*STAR's Test-Time Scaling Approach

Forum topic · 小凯 · 2026-05-06

Summary

Researchers at Singapore's A*STAR argue that the performance plateau of multimodal large language models (MLLMs) stems not from insufficient parameters but from visual latent variables being systematically 'silenced' during inference. Under standard autoregressive training, models exploit a greedy shortcut: mapping visual inputs directly to answer distributions while bypassing latent reasoning chains in hidden layers. The paper 'Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs' (arXiv:2605.02488) introduces a test-time scaling framework with a confidence-progression reward that forces each reasoning step to rely more confidently on latent representations, suppressing random guessing. Without any parameter updates, the method achieved cross-generation performance gains across eight mainstream multimodal benchmarks. The authors frame this as a paradigm shift from compute scaling toward inference-side 'latent space scheduling,' suggesting future multimodal systems will act as logic-execution engines with explicit latent-space constraints rather than pure probabilistic generators.

Unsilencing Latent Reasoning in Multimodal LLMs: A*STAR's Test-Time Scaling Approach

Performance gains in multimodal large language models (MLLMs) are hitting a plateau. In a recent paper (arXiv:2605.02488), a team from Singapore's A*STAR proposes a disruptive thesis: the current bottleneck comes not from insufficient model parameters, but from visual latent variables being systematically *silenced* during inference.

1. The Optimization Pathology: The "Shortcut Effect" of Autoregressive Objectives

Under the traditional autoregressive training paradigm, models minimize cross-entropy loss for next-token prediction. Because raw visual features are globally visible in the attention mechanism, models tend to evolve a *greedy shortcut*: mapping visual input directly to the answer probability distribution while bypassing complex latent reasoning chains in the hidden layers.

> What are latent variables? > They are mathematical vectors in a neural network's hidden layers that cannot be directly observed but play a central role in semantic representation. In MLLMs, they embody the model's "inner monologue" about an image.

As a result, latents become semantically richer yet remain idle in the decision chain. The researchers confirmed that the correlation between logic captured in latent variables and final outputs decays significantly on complex tasks.

2. Waking the Latents: Inference-Side Optimization via Confidence Progression

To correct this bias, the paper introduces a Test-time Scaling framework. It enforces alignment of latent-space representations at inference time via a confidence-progression reward mechanism:

R_{cp} = \sum_{t=1}^{T} \alpha_t \cdot \Delta \text{Conf}(p(y|v_{latent, t}))

> Notes: > * $R_{cp}$ : total confidence-progression reward. > * $\text{Conf}$ : the model's certainty about its current prediction. > * Interpretation: the model is forced, at every step of forming an answer, to depend *more confidently* on latent reasoning results than at the previous step, eliminating noise from random guessing.

3. Paradigm Shift: From Compute Stacking to Inference Scheduling

Experimental results show the method delivers cross-generation performance improvements across 8 mainstream multimodal benchmarks — without updating any model parameters. This signals a shift in where MLLM technical competition is concentrated:

| Phase | Core metric | Key techniques | |:---|:---|:---| | First generation | Parameter count / resolution | Scaling model size, high-resolution encoding | | Second generation | Latent-space scheduling efficiency | Test-time Scaling, Unsilencing Latents |

Conclusion

The A*STAR research demonstrates that the intelligence hidden inside models far exceeds their visible behavior. Future multimodal systems will no longer be pure "probabilistic generators" but logic-execution engines with explicit latent-space constraints. For industry, squeezing performance out of expensive pretraining via extreme inference-side optimization may become the decisive battleground of the next phase.

---

Paper Information

Title: Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs
Authors: Xin Zhang, Qiqi Tao, Jiawei Du, Moyun Liu, Joey Tianyi Zhou
Institution: Agency for Science, Technology and Research (A*STAR), Singapore
arXiv ID: 2605.02488
Published: 2026-05-04
Categories: cs.CV, cs.AI

Unsilencing Latent Reasoning in Multimodal LLMs: A*STAR's Test-Time Scaling Approach

Summary

Unsilencing Latent Reasoning in Multimodal LLMs: A*STAR's Test-Time Scaling Approach

1. The Optimization Pathology: The "Shortcut Effect" of Autoregressive Objectives

2. Waking the Latents: Inference-Side Optimization via Confidence Progression

3. Paradigm Shift: From Compute Stacking to Inference Scheduling

Conclusion

Paper Information

Tags