Model Releases
[Benchmark] Optimal DFlash2 quants for speed and context size, 5090 RTX, llama.cpp, Qwen 3.8 27B Dynamic3 Unsloth. Comparison with MTP
Graph: A cumulative metric of Speed x Context Size - the higher the dot - the better. Helpful for looking for the most balanced solution. The research relies on DFlash2 Q2 work by AnalogAlok: https://
Graph: A cumulative metric of Speed x Context Size - the higher the dot - the better. Helpful for looking for the most balanced solution. The research relies on DFlash2 Q2 work by AnalogAlok: https://huggingface.co/analogalok/Qwen3.8-27B-DFlash2-Q2_K-GGUF TLDR: Q4 quants for DFlash2 demonstrate the highest speeds due to their higher acceptance rate. Q2 free up so much context that they are comparable with MTP in that sense, but can still be tuned to perform 10-20% faster. Q8 don't have any upsides and should be avoided as they consume too much context and have lower peaks than Q4. There seems to be no real upside in using MTP as DFlash2 Q2 at n=1 and n=2 demonstrate comparable speeds and context size. Overall Winners: Best balance: DFlash2 Q2, n-max 5 = 137.19 tps with context size 121k. Top speed: DFlash2 Q4, n-max 7 = 154.08 tps with context size 102k. Largest context size: MTP n-max 1 = 85.77 tps n-max 1 at 131k. Tables with the outcomes per quant and per number of drafts below. Speed measured as an average of 9 runs at inputs of 65k tokens. For brevity for DFlash2 only n-max 5+ are shown. Q2 requested n-max Speed at 65k input Acceptance Mean span Maximum context First OOM 5 137.19 t/s 53.29% 3.67 121,600 121,728 6 139.39 t/s 47.83% 3.89 117,504 117,632 7 139.17 t/s 42.63% 4.01 113,664 113,728 8 138.71 t/s 42.63% 4.01 109,568 109,696 Q4 requested n-max Speed at 65k input Acceptance Mean span Maximum context First OOM 5 146.02 t/s 58.23% 3.91 110,592 110,656 6 152.52 t/s 54.17% 4.24 106,496 106,624 7 154.08 t/s 48.71% 4.42 102,400 102,528 8 153.45 t/s 48.71% 4.42 98,304 98,432 Q8 requested n-max Speed at 65k input Acceptance Mean span Maximum context First OOM 5 144.31 t/s 57.96% 3.89 86,784 86,848 6 151.61 t/s 54.20% 4.25 82,944 83,072 7 153.46 t/s 48.90% 4.44 78,848 79,040 8 ≈ n7* ≈ n7* ≈ n7* 74,752 74,880 Now for the MTP results and the respective winners: MTP drafts Generation speed Acceptance Mean accepted span Maximum context First OOM Context × speed 1 (top context) 85.77 t/s 87.13% 1.87 131,328 131,456 11.26M 2 109.39 t/s 79.17% 2.58 128,000 128,128 14.00M 3 (best balance) 121.62 t/s 69.97% 3.09 124,416 124,544 15.13M 4 124.39 t/s 62.03% 3.47 120,832 120,960 15.03M 5 125.92 t/s 55.97% 3.79 117,504 117,632 14.80M 6 (top speed) 128.32 t/s 50.09%. 4.00 113,920 114,048 14.618M 7 128.28 t/s 46.50% 4.25 110,592 110,720 14.187M 8 112.58 t/s 40.45% 4.24 106,752 106,880 12.018M Tested config: llama-server --model unsloth-qwen38-27b-dynamic3/Qwen3.8-27B-UD-Q8_K_L.gguf --mmproj unsloth-qwen38-27b/mmproj-F16.gguf --no-mmproj-offload --image-min-tokens 1024 --alias qwen3.8-27b-q8-0 --ctx-size 113920 --gpu-layers all --fit off --parallel 1 --batch-size 512 --ubatch-size 128 --cache-type-k q8_0 --cache-type-v q8_0 --kv-offload --kv-unified --cache-ram 8192 --cache-idle-slots --flash-attn on --jinja --reasoning on --reasoning-preserve --spec-type draft-mtp --spec-draft-n-max 6 --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 --spec-draft-ngl all --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0 --repeat-penalty 1.0 --timeout 3600 submitted by /u/Opening-Broccoli9190 [link] [comments]
Related
- DFlash 2 available for Qwen 3.8 27B and Muse Glimmer
- Local agentic coding Benchmark : Qwen 3.8 27B (in many weights quants / cache quants / engine / reasoning effort) vs others.
- Qwen 3.6 27B flags/settings in llama.cpp
- I benchmark DFlash 2 (PR build) in llama.cpp on Qwen 3.8 27B against all speculative methods for 3 days. 2.26x on 100 real coding prompts, 4.68x with one n-gram drafter on top. Up to 8x on specific cases.
Source: r/LocalLLaMA | 2026-08-24