Updated my benchmark with a new vLLM based recipe for Qwen 3.8 Flash Next : now up to 98/100 (instead of 91 previously)

I was using: weights https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4 with the optimized SGLANG (patched) from https://old.reddit.com/r/BlackwellPerformance/comments/1w04xb7/qwen38_flashnext_on_1x_rtx_pro_6000_171_ts_c1_428/ Now I'm using: weights (AWQ W4A16) from: https://huggingface.co/wtdcode/Qwen3.8-Flash-Next-AWQ-W4A16 PLE (INT4) from: https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant with vLLM patched with the patch from the same repo cf https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant#serve the goal was to load both the weights and the n-gram PLE quantized in 4bits either on my sm89 or sm120 devices It's way slower (for my low concurrency usecase) but also a lot better. I was surprised to see such a delta. I reached 98/100 (instead of 91) both with medium and xhigh reasoning (still not useful for this bench). And now it really feels like a huge setup up from the other models. It's the best score AND the most efficient... I'll try to dig deeper to understand if the difference comes from the engine (and its patches) or the quants themselves. And try to optimize further the vLLM receipt for my setup as always, the graphs and the data : here and here submitted by /u/WonderRico [link] [comments]

Source: r/LocalLLaMA | 2026-09-04

Related