Model Releases
Don't Sleep on EXL3 Quants
I'm running Muse Glimmer 30B EXL3-SC 3.00bpw H4, fully resident on my 12GB VRAM GPU at 100K context with Q8_O KV cache. It's a joy to use a dense 30B model at this size and still get 30 tok/s on a VRA
I'm running Muse Glimmer 30B EXL3-SC 3.00bpw H4, fully resident on my 12GB VRAM GPU at 100K context with Q8_O KV cache. It's a joy to use a dense 30B model at this size and still get 30 tok/s on a VRAM-constrained laptop. It's supposed to be only slightly worse than the official 17GB K-quant at a much smaller footprint, and for my Hermes Agent use case I don't notice a quality difference. It's just much faster. I've tried Qwen 3.8 27B at SC2.20bpw H3 too. Definitely usable but I'm sticking with Unsloth UD_Q4_K_XL for Qwen 3.8 27B because it's mainly for coding. submitted by /u/PyaesoneP [link] [comments]
Related
- Tested Muse Glimmer locally on coding with OpenCode & agentic work
- DFlash 2 available for Qwen 3.8 27B and Muse Glimmer
- [[benchmark-optimal-dflash2-quants-for-speed-and-context-size-|[Benchmark] Optimal DFlash2 quants for speed and context size, 5090 RTX, llama.cpp, Qwen 3.8 27B Dynamic3 Unsloth. Comparison with MTP]]
- New Muse-Glimmer-30B SoTA Quants - hopefully a new lineup :)
Source: r/LocalLLaMA | 2026-08-30