Model Releases
We can't thank llama.cpp and @ggerganov enough for being among the pioneers of local AI. But we sure as hell can contribute back! Upstream c…
We can't thank llama.cpp and @ggerganov enough for being among the pioneers of local AI. But we sure as hell can contribute back! Upstream contributions will now become a priority in @qvac. One of our
We can't thank llama.cpp and @ggerganov enough for being among the pioneers of local AI. But we sure as hell can contribute back! Upstream contributions will now become a priority in @qvac. One of our PR just got merged into llama.cpp: new Metal kernels for speculative decoding on Apple Silicon. Before this PR, speculative decoding on a Mac was slower than plain decoding. On an M3 Ultra it now runs up to 3.4x faster than plain decoding, 110 tok/s against 32.1. Thank…
Related
- llama.cpp recently added DFlash support to its speculative decoding arsenal. Along with MTP, Eagle3 and various ngram-based techniques, the …
- Local AI just got faster. ⚡️ We worked with @ggerganov to add DFlash support in llama.cpp delivering ~2x faster inference.
Source: Georgi Gerganov (X) | 2026-10-05