Muse-Glimmer 30B Hits ~280 t/s in Real Production Coding

These numbers were captured during a real feature implementation task in Next.js and Nest.js (adding a theme switching system across components). The structural predictability of UI/state refactoring is likely why DFlash hit such a high draft acceptance rate (~97%). Here is a quick log analysis and performance summary running Muse-Glimmer-30B (UD- Q6_K_XL) paired with DFlash (Speculative Decoding) via llama.cpp (llama-server + single RTX 5090). -ngl 99 -c 200000 --host 0.0.0.0 --port 8080 --timeout 600 --cache-reuse 256 --parallel 1 --flash-attn on --spec-type draft-dflash --spec-draft-n-max 16 --spec-draft-p-min 0.7 --spec-draft-ngl 99 --cache-type-k q8_0 --cache-type-v q8_0 --no-webui --load-mode none --cache-ram 12192 --temperature 0.8 --top-k 30 --top-p 0.95 --min-p 0.05 --repeat-penalty 1.1 --repeat-last-n 64 --reasoning on --chat-template-kwargs {"enable_thinking":true} Compared to Qwen 3.6 27B: No Chinese language-mixing bugs, no overthinking loops, and concise responses. Its lighter memory footprint at Q6 also freed up more VRAM/RAM for a much larger context size. Metric Measured Value Notes Generation Speed (Peak) 100 – 287 tokens/sec Average ~173 t/s across all tasks; ideal for IDE completion Prompt Processing (Short) 1,000 – 2,100 t/s ~100–1,000 tokens evaluated in 0.1s – 0.5s Prompt Processing (Long) 2,654 tokens/sec 14.3k tokens evaluated in ~5.35 seconds Draft Acceptance Rate 63.3% – 97.5% Average 82.3% across 22 test tasks Mean Accepted Draft Length Up to 14.0 tokens Multi-token acceptance driving massive acceleration KV Cache Reuse (LCP) 99.9% (f_sim = 0.999) Near-instant response on repeated code contexts submitted by /u/Ok-Shower7286 [link] [comments]

Source: r/LocalLLaMA | 2026-08-11

Related