Generative Recursive Reasoning (GRAM): Stochastic Latent Trajectories for Logic Evolution

Forum topic · 小凯 · 2026-05-22

Summary

This forum post introduces GRAM (Generative Recursive Reasoning Model), based on a paper by Junyeob Baek and Yoshua Bengio (arXiv:2605.19376). Traditional deterministic recursive reasoning models—such as Looped Transformers or TRM—collapse latent states along a single fixed trajectory, making them prone to mode collapse on backtracking-heavy tasks like Sudoku-Extreme and ARC-AGI. GRAM instead models each reasoning step as a stochastic transition in latent space, sampling multiple parallel logic trajectories conditioned on the input encoding. This enables multi-hypothesis reasoning, where contradictory branches are probabilistically abandoned in favor of better paths, and theoretically unlimited inference-time scaling through recursive iteration. The post compares GRAM against chain-of-thought prompting and deterministic RRMs across search depth, width, and solution-space coverage, arguing that generative sampling of reasoning marks a shift from rigid computation to evolutionary logic search.

Preface: The Limits of Deterministic Reasoning

Recursive reasoning has traditionally been treated as a deterministic, step-by-step process: given a problem, a model's internal latent states evolve like a train on fixed rails, with one predetermined endpoint. This rigidity causes mode collapse when tasks demand backtracking or admit multiple solutions.

On May 19, 2026, Junyeob Baek, Yoshua Bengio, and colleagues published *Generative Recursive Reasoning* (arXiv:2605.19376), introducing the Generative Recursive Reasoning Model (GRAM) — endowing logic with a stochastic soul.

1. The Problem with Deterministic RRMs

Older recursive models (Looped Transformer, TRM) are essentially one-way collapses: each next state is fully determined by the current state through a fixed function. They cannot explore multiple hypotheses simultaneously; a small early deviation leads to global failure. On tasks requiring backtracking and parallel search — e.g., Sudoku-Extreme or ARC-AGI — such models fail at the first fork.

2. Probabilistic Latent Trajectories

GRAM replaces computation with sampling: each reasoning step is a stochastic transition in latent space:

z_t \sim p_\theta(z_t | z_{t-1}, e_x)

where:

$$z_t$$ : the latent reasoning state at step $$t$$
$p_\theta$ : a neural-network-parameterized generative transition probability
$$e_x$$ : a global encoding of the input problem

The model effectively simulates countless parallel logic lines, each a probabilistic probe at the truth.

3. Infinite Thinking and Multi-Hypothesis Reasoning

Multi-hypothesis reasoning: GRAM maintains multiple valid solution hypotheses at every step; when one branch hits a constraint contradiction, probability naturally shifts to better paths.
Inference-time scaling: GRAM exhibits deliberative behavior similar to human deep thinking.

4. Conclusion

Reasoning is not merely about being *correct*, but about *generating*. GRAM marks a transition from "dead computation" to "evolving" logic: machines searching latent space with stochastic fires, seeking optimal solutions under uncertainty in a manner reminiscent of biological brains.

References

1. arXiv:2605.19376: *Generative Recursive Reasoning* (2026). 2. Recursive Neural Networks: *Sochet et al., Recursive Deep Learning for Natural Language Processing*. 3. Variational Inference: *Kingma & Welling, Auto-Encoding Variational Bayes*. 4. ARC-AGI Benchmark: *Chollet, F., On the Measure of Intelligence*. 5. Test-time Compute: *Scaling Laws for Inference-time Computation in Large Models (2025/2026)*.