Research
Arbitrage: Efficient Reasoning via Advantage-Aware Speculation
Modern Large Language Models achieve impressive reasoning capabilities with long Chain of Thoughts, but they incur substantial computational cost during inference, and this motivates techniques to imp
Modern Large Language Models achieve impressive reasoning capabilities with long Chain of Thoughts, but they incur substantial computational cost during inference, and this motivates techniques to improve the performance-cost ratio. Among these techniques, Speculative Decoding accelerates inference by employing a fast but inaccurate draft model to auto-regressively propose tokens, which are then verified in parallel by a more capable target model. However, due to unnecessary rejections caused by token mismatches in semantically equivalent steps, traditional token-level Speculative Decoding…
Related
- Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective
- Adaptive Thinking: Large Language Models Know When to Think in Latent Space
- CoVSpec: Efficient Device-Edge Co-Inference for Vision-Language Models via Speculative Decoding
- LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification
Source: Apple ML Research | 2026-08-07