Research
Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
Discrete flow matching generates text by iteratively transforming noise tokens into coherent language, but may require hundreds of forward passes. Distillation uses the multi-step trajectory to train
Discrete flow matching generates text by iteratively transforming noise tokens into coherent language, but may require hundreds of forward passes. Distillation uses the multi-step trajectory to train a student to reproduce the process in a few steps. When the student underperforms, the usual explanation is insufficient capacity. We argue the opposite: the trajectory is the bottleneck, not the student. Each training trajectory is built through a chain of blind stochastic jumps with no evaluation of sequence quality; a single bad decision at an early midpoint propagates through subsequent steps…
Related
- Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations
- A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models
- Few-Step Diffusion Language Models via Trajectory Self-Distillation
Source: Apple ML Research | 2026-09-16