Model Releases
I tested different Qwen 3.8 27B quants
There is a lot of discussion which quant to use. Especially a lot of post how much speed you gain with FP4, but not much about quality. So i tested some of the models that I use on my R9700. Models: Q
There is a lot of discussion which quant to use. Especially a lot of post how much speed you gain with FP4, but not much about quality. So i tested some of the models that I use on my R9700. Models: Qwen3.8-27B-UD-Q6_K_L.gguf Swift-Qwen3.8-27B-Q6_K.gguf Swift-1.5-Qwen3.8-27B-MXFP4-A-outQ6_K.gguf Hypotheses: Qwen3.8-27B-UD-Q6_K_L.gguf best in accuracy and slowest Swift-Qwen3.8-27B-Q6_K.gguf second best in accuracy and speed Swift-1.5-Qwen3.8-27B-MXFP4-A-outQ6_K.gguf worst in accuracy and fastest Servers for the models: Qwen3.8-27B-UD-Q6_K_L.gguf and Swift-Qwen3.8-27B-Q6_K.gguf: Llama.cpp Swift-1.5-Qwen3.8-27B-MXFP4-A-outQ6_K.gguf: whirl Environment: The Pi agent set up gsm8k (math) and humaneval (code) tests within the same docker container. Results: I was surprised that the MXFP4 quant was the best in math and Qwen3.8-27B-UD-Q6_K_L the worst. Also that Swift-Qwen3.8-27B-Q6_K was the slowest. For better more precise results I would need to run whole test suits and run them multiple times. But that would take to much time on my hardware. Can anyone explain how is MXFP4 is first in math and last in code? submitted by /u/JurBank [link] [comments]
Related
- Compared Qwen 3.8 27B community quants on RTX 6000 vs Claude Opus 4.6
- Qwen 3.8 27b is strong even at Q3_xxs
- Perhaps the highest quality mainline quants of Qwen3.8 27B?
Source: r/LocalLLaMA | 2026-10-11