Research
Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps
arXiv:2610.00497v1 Announce Type: cross Abstract: We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive lang
arXiv:2610.00497v1 Announce Type: cross Abstract: We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive language (AR) model. We theoretically demonstrate that the coupling between Gumbel noise and one-hot token sequences induced by an autoregressive model yields non-intersecting linear paths connecting the noise to the sequence representations. To further enhance high-quality few-step path sampling, we use a flow map semigroup objective where the tangent (velocity) condition is guided directly by the AR teacher. Across various benchmarks, including pretraining and downstream tasks, GSF can outperform current few-step language generation baselines.
Related
- ELF: Embedded Language Flows
- Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Models
- Flow-Map GRPO: Reinforcement Learning for Few-Step Flow-Map Generators via Anchored Stochastic Composition
Source: arXiv cs.AI | 2026-10-02