Local Ai
Qwen3.8-Flash-Next @ 1.66bpw-exl3 STABLE under 36gb
Fits entirely in a 5090, no offloaded experts. Super-solid sub-2bpw! (Liquid cat demo video generated by the quantized model) From the Cat's mouth: Stable inference throughout, no reasoning loops, jus
Fits entirely in a 5090, no offloaded experts. Super-solid sub-2bpw! (Liquid cat demo video generated by the quantized model) From the Cat's mouth: Stable inference throughout, no reasoning loops, just smooth sailing at 200-300 t/s (and a lot of compacting since one 32 GB GPU still doesn't leave a lot of room for context, even with cache quantization. https://huggingface.co/turboderp/Qwen3.8-Flash-Next-exl3 submitted by /u/Unstable_Llama [link] [comments]
Source: r/LocalLLaMA | 2026-10-21