You know that feeling during an exam when a vague answer flashes through your mind — you're not sure it's right, but following that thread leads you to the correct answer?
Diffusion language models have a similar experience. Except they throw away those "uncertain answers" directly.
Until now, no one realized: those discarded answers are the most valuable.
Diffusion Models' "Scrap Yard"
Some background first. Diffusion language models work completely differently from GPT. GPT generates token by token, while diffusion models first "mask" an entire passage — turning every position into noise — then denoise step by step, predicting all positions simultaneously at each step.
Here's the key: at each denoising step, the model makes a prediction for every position, but only keeps "high-confidence" tokens. Low-confidence ones are discarded and re-guessed at the next step.
It's like a student filling in all the blanks on a test with their best guesses, then keeping only the ones they're confident about and erasing the rest to try again.
SARDI's core finding: those erased answers often already contain critical information.
Which City Holds the Mona Lisa?
The paper uses an elegant example. Suppose the question is: "What city houses the museum that holds the Mona Lisa?"
This requires two-step reasoning: 1. The Mona Lisa is in the Louvre 2. The Louvre is in Paris
Traditional RAG can only retrieve using the question itself, but the question contains neither "Louvre" nor "Paris" — the first retrieval step gets stuck.
Meanwhile, during the diffusion model's denoising process, even before the final answer is settled, intermediate states may already surface the word "Louvre" — its confidence isn't high enough to make it into the final output, but it's already sufficient for retrieval!
Searching with "Louvre" immediately finds the evidence "the Louvre is in Paris." That's SARDI's core idea: use low-confidence tokens from intermediate states as retrieval queries, letting the model "peek at its own future."
Two Findings, One Framework
SARDI is built on two empirical findings:
Finding one: intermediate states are natural "look-ahead signals." In multi-hop QA, diffusion models' intermediate denoising states surface bridge entities earlier than autoregressive models. An autoregressive model must first write out "Louvre" before it can retrieve the next step, while a diffusion model "senses" it before deciding whether to commit it.
Finding two: RAG makes parallel decoding feasible. Diffusion models have long had a pain point — predicting multiple tokens simultaneously tends to produce conflicts. But the paper found that once retrieved evidence is available as context, many output tokens are directly copied or adapted from the evidence, becoming conditionally independent of each other, greatly reducing conflicts.
The two findings fit together perfectly: intermediate states provide better retrieval signals → retrieved evidence lowers inter-token dependencies → parallel decoding flows more smoothly → faster generation of intermediate states → even better retrieval.
SARDI interleaves retrieval with denoising in this positive loop: after each denoising step, it builds a query from the partially denoised sequence, retrieves new evidence, and continues denoising based on the updated context.
Results: Faster and Better
Across five multi-hop QA benchmarks, SARDI leads across the quality-latency frontier:
- Compared to training-free diffusion baselines, SARDI achieves higher quality at the same latency
- Compared to training-free autoregressive baselines (including FLARE), SARDI reaches equal or better quality at lower latency
- On the MuSiQue dataset, SARDI achieves accuracy comparable to FLARE with less than half the latency
Why Does This Matter?
SARDI's significance goes beyond "yet another RAG method." It reveals something deeper: the architectural properties of diffusion language models — parallel prediction, progressive denoising — are not just a speed advantage, but an information advantage.
An autoregressive model's information flow is strictly left-to-right; it can only decide the next step based on what it has already written. A diffusion model's information flow is global and parallel — every position can "sense" the tendencies of other positions, even when those tendencies aren't yet settled.
It's like a team solving a problem: the autoregressive model is one person writing step by step from start to finish, while the diffusion model is an entire team working on different positions simultaneously, then consolidating everyone's confident parts. SARDI's contribution — it lets team members who are "uncertain but possibly right" also provide clues for the team's decisions.
Those discarded tokens aren't scrap. They're prophets.
---
📄 Paper: Self-Augmenting Retrieval for Diffusion Language Models 💻 Code: github.com/pauljngr/SARDI