Local Ai
Qwen3.8 Flash AP Quants
Quite surprised to be beating other high quality quants. It took a lot of benchmarking to get here and we are quite pleased with these, hope they are useful to the community. It required a modified wa
Quite surprised to be beating other high quality quants. It took a lot of benchmarking to get here and we are quite pleased with these, hope they are useful to the community. It required a modified way of measuring KLD with a new dataset, since the NGRAM got in the way by remembering basically all of wikipedia. We tried to not only go for high precision, but also keep prefill performance in mind. Full model card here https://huggingface.co/agentionai/Qwen3.8-Flash-Next-AP-GGUF Let us know if there are any issues. https://preview.redd.it/nq48xxtxj6nh1.png?width=1800&format=png&auto=webp&s=5059384113868ed2717e04b24603a5fbb7d9908c submitted by /u/Dutchnamn [link] [comments]
Related
- Qwen3.8-Flash-Next NVFP4 2xDGX Spark config: 50t/s decode, 2,900t/s prefill
- Optimal 1.25 bit quantization of Qwen3.8-Flash-Next
- Unsloth Q1-Q2 Qwen3.8-27B with MTP since the unsloth ones don't ship with for the lowest quants
- I pushed Qwen3.8-27B to 2.000 prefill per second and 132 decode per second on A RTX 3090.
Source: r/LocalLLaMA | 2026-09-02