Summary
This paper presents the first systematic study of masked diffusion language models (MDLMs) for graph-to-text generation. By analyzing MDLM generation trajectories—the order in which tokens are unmasked during iterative decoding—the authors find that, unlike autoregressive LLMs that generate text linearly, MDLMs naturally prioritize entities first, followed by relational and functional words, with structural tokens resolved last. The study also uncovers a previously undocumented failure mode of supervised fine-tuning (SFT): it disrupts this strategy by prematurely anchoring structural end-of-text tokens early in the decoding trajectory, effectively fixing output length and causing information omission or hallucination. To address this, the authors propose lambda-scaled structural decoding, a training-free decoding-time modification that lowers structural-token confidence and restores +9.4 BLEU-4. Finally, they introduce Graph-LLaDA, which integrates a graph transformer encoder into the LLaDA decoding process to explicitly incorporate relational graph structure. Cross-dataset evaluation reveals that prior baselines overfit dataset-specific patterns, while LLM- and MDLM-based approaches generalize significantly better. arXiv: 2605.31564.
Research area: NLP / AI
Authors: Qing Wang, Jacob Devasier, Chengkai Li
Published: 2026-05-29
arXiv: 2605.31564
PDF: 2605.31564.pdf
Summary
This paper presents the first systematic study of masked diffusion language models (MDLMs) applied to graph-to-text generation.
By analyzing MDLM generation trajectories—the order in which tokens are unmasked during iterative decoding—the authors find that, unlike autoregressive LLMs which generate text linearly, MDLMs naturally prioritize entities first, followed by relational words and functional words, with structural tokens resolved last.
A previously undocumented failure mode of supervised fine-tuning (SFT) is also discovered: SFT disrupts this strategy by prematurely anchoring structural end-of-text tokens early in the decoding trajectory, effectively fixing output length, which can lead to information omission or hallucination. To address this, the authors propose lambda-scaled structural decoding, a training-free decoding-time modification that lowers the confidence of structural tokens, recovering +9.4 BLEU-4.
Finally, the paper introduces Graph-LLaDA, which integrates a graph transformer encoder into the LLaDA decoding process to explicitly incorporate relational graph structure. Cross-dataset evaluation reveals that prior baselines overfit dataset-specific patterns, while LLM- and MDLM-based approaches generalize significantly better.
---
*Auto-collected on 2026-06-02*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177980741