Discrete Beckmann Transport Models for One-Step Language Modeling (arXiv 2609.15903)

Forum topic · 小凯 · 2026-09-16

Summary

Discrete diffusion and flow models are promising alternatives to autoregressive language models, but compressing multi-step sampling into fewer steps usually requires distilling a pretrained teacher model, which caps student quality and demands a costly two-stage pipeline. This paper introduces Discrete Beckmann Transport Models (DBTM), built on a time-independent flow whose autonomous transport map provably carries any point in the ambient space to a fixed point on the simplex vertices in a single step. The fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, eliminating the need for a teacher flow and time conditioning. Partially trained maps correspond to finite-time truncations of the flow, so generation reduces to iterating the map to its fixed point, and extending the map to a partial context interpolator turns extra function evaluations into refinement steps. On language modeling and reasoning tasks, DBTM achieves one-step and few-step generation that outperforms discrete diffusion and continuous flow baselines in quality and accuracy. Paper: arXiv 2609.15903.

Paper Overview

Field: Machine Learning
Authors: Sophia Tang, Shiyi Wang
Published: 2026-09-14
arXiv: 2609.15903

Introduction

Discrete diffusion and flow models are a promising alternative to autoregressive language models, but compressing many-step sampling into fewer steps typically requires distilling a pretrained teacher model. This caps the student at the teacher's quality and requires a costly two-stage training pipeline.

Method: Discrete Beckmann Transport Models (DBTM)

DBTM is built on a time-independent flow whose autonomous transport map provably carries any point in the ambient space to a fixed point on the vertices of the simplex in a single step.
This fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, removing the requirement for a teacher flow and time conditioning.
Under this construction, a partially trained map corresponds to a finite-time truncation of the flow, so generation reduces to iterating the map until it reaches a fixed point.
The map is further extended into a partial context interpolator, so additional function evaluations act as refinement steps rather than ODE integration steps.

Results

On language modeling and reasoning tasks, DBTM achieves one-step and few-step generation that outperforms discrete diffusion and continuous flow baselines in both quality and accuracy.