Model Releases
yall are sleeping on qwen 3.8 27b q2 + q2 dflash + q5 kv
ok bit more context: it's actually a QAT Q2 for Qwen 3.8 27 B: https://huggingface.co/sdkyuan/qwen3.8-27B-qat-q2_0-gguf QAT Q2 for DFlash model: https://huggingface.co/HermiHg/Qwen3.8-27B-DFlash2-Q2_K
ok bit more context: it's actually a QAT Q2 for Qwen 3.8 27 B: https://huggingface.co/sdkyuan/qwen3.8-27B-qat-q2_0-gguf QAT Q2 for DFlash model: https://huggingface.co/HermiHg/Qwen3.8-27B-DFlash2-Q2_K_S-MIX-GGUF Q5 KV seems to cause 0 problems for me; I've used it up to 200K tokens of context. Total RAM usage is liek 13-14 ish gb and I've seen very little degradation with the QAT Q2 3.8 27B Its just crazy that these days with a 12gb card (fits if drop down to 100K context) you can run a model thats more capable than sonnet 4.6, genuinely crazy stuff submitted by /u/Square_Light1441 [link] [comments]
Related
- DFlash 2 available for Qwen 3.8 27B and Muse Glimmer
- A preliminary Qwen3.8-27B model card is live!
- [[benchmark-optimal-dflash2-quants-for-speed-and-context-size-|[Benchmark] Optimal DFlash2 quants for speed and context size, 5090 RTX, llama.cpp, Qwen 3.8 27B Dynamic3 Unsloth. Comparison with MTP]]
- Has anyone actually made 64k feel like 300k+ with recursive local agents?
Source: r/LocalLLaMA | 2026-08-28