Model Releases
releasing my local AI slide deck covers from prefill vs decode, MoE vs dense, VRAM vs unified memory, quantization to speculative decoding, …
releasing my local AI slide deck covers from prefill vs decode, MoE vs dense, VRAM vs unified memory, quantization to speculative decoding, everything with llama.cpp feel free to reuse with attributio
releasing my local AI slide deck covers from prefill vs decode, MoE vs dense, VRAM vs unified memory, quantization to speculative decoding, everything with llama.cpp feel free to reuse with attribution
Related
- llama.cpp recently added DFlash support to its speculative decoding arsenal. Along with MTP, Eagle3 and various ngram-based techniques, the …
- More info about speculative decoding with llama.cpp: https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md
- Overclocking DDR5 For Faster MoE Prefill and Decode
- Qwen3.8-Flash-Next (125B) on a single Strix Halo mini PC: 44-59 tok/s with speculative decoding, ~1,400 tok/s prefill, engine is open
- Local AI just got faster. ⚡️ We worked with @ggerganov to add DFlash support in llama.cpp delivering ~2x faster inference.
Source: Georgi Gerganov (X) | 2026-10-06