GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring

arXiv:2608.20375v1 Announce Type: new Abstract: Tree-based speculative decoding raises the mean accepted tokens of standard speculative decoding by verifying multiple draft paths, and existing tree builders typically construct these paths through parent-conditioned expansion, where each child token is generated conditioned on its parent path. This construction is incompatible with diffusion language model (DLM) drafters such as DFlash, which produces all future-position distributions in a single forward pass. DDTree bridges this gap by treating high-probability tokens from each future-position distribution as candidate nodes and selecting edges between consecutive positions under a fixed node budget. However, its edge selection relies on token probability alone without modeling parent--child compatibility, so target-compatible tokens can be attached to wrong parents; moreover, its fixed budget ignores that the throughput-optimal tree size varies with the decoding state. We propose GRAFT, a draft-tree construction framework for DLM-based speculative decoding. GRAFT introduces Target-Distilled Edge Scoring (TDES), which distills parent--child preferences from target-model traces to select target-compatible edges, and State-Aware Budget Allocation (SABA), which sets the per-round tree budget by balancing expected draft gain against verification cost. Across multiple models and tasks, GRAFT achieves 2.13imes--6.36imes end-to-end speedup over autoregressive decoding while adding less than 0.5,ms of overhead per round, approximately 1.4% of the target-model verification latency.

Related

• Accelerating Speculative Decoding with Block Diffusion Draft Trees
• SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting
• Cost-Aware Diffusion Draft Trees for Speculative Decoding
• D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models

Source: arXiv cs.CL | 2026-08-24