Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism

arXiv:2601.05524v2 Announce Type: replace Abstract: Parallel Speculative Decoding (PSD) accelerates traditional Speculative Decoding (SD) by overlapping draft generation with verification. However, it remains hampered by two fundamental challenges: (1) a theoretical speedup ceiling dictated by the speed ratio between the draft and target models, and (2) high computational waste and pipeline stall due to mid-sequence token rejections of early errors. To address these limitations, we introduce extsc{Double} (Double Retrieval Speculative Parallelism). By bridging the gap between SD and PSD, our framework resolves the Retrieval Precision-Efficiency Dilemma through a novel synchronous mechanism. Specifically, we enable the draft model to execute iterative retrieval speculations to break the theoretical speedup limits; to alleviate rejections without rollback, the target model performs authoritative retrieval to generate multi-token guidance. extsc{Double} is entirely training-free and lossless. Extensive experiments demonstrate state-of-the-art speedup of extbf{5.3}imes on LLaMA3.3-70B and extbf{2.8}imes on Qwen3-32B, significantly outperforming the advanced method EAGLE-3 that requires extensive model training.

Related

• DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification
• See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs
• LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
• DYCP: Dynamic Context Pruning for Long-Form Dialogue with LLMs

Source: arXiv cs.CL | 2026-04-10