Summary
Discrete diffusion and flow models are promising alternatives to autoregressive language models, but compressing multi-step sampling into fewer steps typically requires distilling a pretrained teacher model, capping the student's quality and demanding a costly two-stage pipeline. This paper introduces Discrete Beckmann Transport Models (DBTM), built on a time-independent flow whose autonomous transport map provably carries any point in the ambient space to a fixed point on the simplex vertices in a single step. The fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, eliminating the need for a teacher flow and time conditioning. A partially trained map corresponds to a finite-time truncation of the flow, so generation reduces to iterating the map to its fixed point. The map is further extended to a partial context interpolator, allowing extra function evaluations to act as refinement steps rather than ODE integration steps. On language modeling and reasoning tasks, DBTM achieves one-step and few-step generation with quality and accuracy surpassing discrete diffusion and continuous flow baselines. arXiv: 2609.15903.
Paper Overview
Field: Machine Learning
Authors: Sophia Tang, Shiyi Wang
Published: 2026-09-14
arXiv: 2609.15903
Abstract
Discrete diffusion and flow models are a promising alternative to autoregressive language models, but compressing many-step sampling into fewer steps typically requires distilling a pretrained teacher model. This caps the student at the teacher's quality and requires a costly two-stage training pipeline.
The authors introduce Discrete Beckmann Transport Models (DBTM), built on a time-independent flow whose autonomous transport map provably carries any point in the ambient space to a fixed point on the vertices of the simplex in a single step. Key contributions:
- The fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, removing the requirement for a teacher flow and time conditioning.
- Under this construction, a partially trained map corresponds to a finite-time truncation of the flow, so generation reduces to iterating a map until reaching the fixed point.
- The map is extended to a partial context interpolator, so additional function evaluations act as refinement steps rather than ODE integration steps.
On language modeling and reasoning tasks, DBTM achieves one-step and few-step generation, outperforming discrete diffusion and continuous flow baselines in both quality and accuracy.
---
*Auto-collected on 2026-09-16*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178634876