Model Releases
New: Llama.cpp adaptive speculation for faster inference
We have been working on some performance optimisations for Qwen3.8 and other models. The main new feature that we introduced is adaptive speculation for Llama.cpp What is it? MTP and DFlash work well
We have been working on some performance optimisations for Qwen3.8 and other models. The main new feature that we introduced is adaptive speculation for Llama.cpp What is it? MTP and DFlash work well to speed up inference work, especially for dense models. However, different content types need different settings. Llama.cpp only supports a single value. This fork introduces adaptive speculation. You set the minimum and maximum and the engine will adjust the number of tokens that are suggested automatically. This leads to improvements in token generation by up to 50% over mainline, especially in Qwen3.8. On a Strix Halo this improved generation from 44t/s to 65t/s for structured content. Github: https://github.com/LaurentZuijdwijk/llama.cpp Release: https://github.com/LaurentZuijdwijk/llama.cpp/releases submitted by /u/Dutchnamn [link] [comments]
Related
- Extremely slow DSpark draft model performance (1-2 t/s) with DeepSeek-V4-Flash on llama-server compared to MTP?
- Tested Nemotron 3.5 Lightning locally on coding, Hermes Agent and agentic work
- llama.cpp PR reports up to 169% faster quantized-KV decode at 118K context on Intel Battlemage from one SYCL kernel switch
Source: r/LocalLLaMA | 2026-08-25