DLM-One: Diffusion Language Models for One-Step Sequence Generation

arXiv:2506.00290v2 Announce Type: replace Abstract: This paper introduces DLM-One, a score-distillation-based framework for one-step sequence generation with continuous diffusion language models (DLMs). DLM-One eliminates iterative refinement by aligning the scores of a student model's outputs with the score function of a pretrained teacher DLM in the forward-diffused noisy space. We demonstrate that our framework is architecture-agnostic and robust across diverse continuous manifolds, including standard token embedding spaces and logit simplex spaces. Through experiments on multiple representative DLMs, we show that DLM-One achieves up to sim2000imes speedup in sampling steps and sim500imes in wall-clock time, while maintaining competitive performance on benchmark text generation tasks. We further analyze failure modes in language-domain diffusion distillation and propose an adversarially-regularized two-stage training scheme to prevent student degeneration. Our findings position one-step score distillation as a viable path for the efficient deployment of continuous diffusion models operating in continuous space for natural language processing.

Related

• Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion
• CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
• MemDLM: Memory-Enhanced DLM Training
• GIFT: Guided Importance-Aware Fine-Tuning for Diffusion Language Models

Source: arXiv cs.CL | 2026-09-03