RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding

arXiv:2604.14885v1 Announce Type: new Abstract: Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency. Speculative decoding (SD) mitigates this through a guess-and-verify strategy, but existing training-free variants face trade-offs: retrieval-based drafts break when no exact match exists, while logits-based drafts lack structural guidance. We propose extbf{RACER} (extbf{R}etrieval-extbf{A}ugmented extbf{C}ontextbf{e}xtual extbf{R}apid Speculative Decoding), a lightweight and training-free method that integrates retrieved exact patterns with logit-driven future cues. This unification supplies both reliable anchors and flexible extrapolation, yielding richer speculative drafts. Experiments on Spec-Bench, HumanEval, and MGSM-ZH demonstrate that RACER consistently accelerates inference, achieving more than 2imes speedup over autoregressive decoding, and outperforms prior training-free methods, offering a scalable, plug-and-play solution for efficient LLM decoding. Our source code is available at href{https://github.com/hkr04/RACER}{https://github.com/hkr04/RACER}.

Related

• Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation
• From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning
• Accelerating Speculative Decoding with Block Diffusion Draft Trees
• SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration
• DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification

Source: arXiv cs.CL | 2026-04-17