Model Releases
Warning: llama.cpp --lazy-mode default changed to auto - large tables may stay on disk
With b10726, the default --lazy-mode change keeps the 51B-parameter PLE n-gram embedding table of Qwen 3.8 Flash Next on disk: it is mmap'd and its rows are read on demand during inference, even with
With b10726, the default --lazy-mode change keeps the 51B-parameter PLE n-gram embedding table of Qwen 3.8 Flash Next on disk: it is mmap'd and its rows are read on demand during inference, even with --load-mode none. It is no longer loaded into RAM unless --lazy-mode off is passed. This change resulted in 50% pp speed penalty, and 15% token generation speed penalty for me. Make sure to add the flag --lazy-mode off if you have enough RAM like me. submitted by /u/whiteh4cker [link] [comments]
Related
- Qwen3.8-Flash-Next (UD-IQ4_XS) on 2x RTX 3060 + 7800X3D, from initial 36 tps prefill to 400 tps and other benchmarks (-sm tensor trap) + VRAM/RAM usage
- Does MTP head get loaded in VRAM by default?
Source: r/LocalLLaMA | 2026-08-31