Model Releases
48Gb VRAM speed AND quality ! (Qwen 3.8 27B Swift 1.5 W8A16)
Hi, Because sometimes you need both speed AND quality, I made my own Qwen 3.8 27B Swift 1.5 quant. It's an 8 bits AWQ + GPTQ quant. Made with love and precision (read the model card) on my 4 RTX 3090,
Hi, Because sometimes you need both speed AND quality, I made my own Qwen 3.8 27B Swift 1.5 quant. It's an 8 bits AWQ + GPTQ quant. Made with love and precision (read the model card) on my 4 RTX 3090, I believe it's the best compromise you can actually get on 48Gb of VRAM. Running it with vLLM and MTP on 2 RTX 3090 in my workflows and it's working flawlessly at almost 100 tok/s decode (with MTP=5), while being less an overthinker than the OG 3.8 27B :) Around 21.6Gb used per card (vLLM launched with 128k context and FP16 KV cache). https://huggingface.co/TacGibs/Swift-1.5-Qwen3.8-27B-W8A16 Hope it'll be usefull to some else ! submitted by /u/TacGibs [link] [comments]
Related
- Swift 1.5 Qwen3.8 27b (A must-have for low thinking!)
- You can offload most of Qwen3.8-Flash-Next's KV cache to RAM with little decode slowdown
- Splash 1.1.0 released, GGUF quants support, MLX import and more
- Qwen 3.8 Flash Next q4_k_m, 130k context, q8 cache on 16GB VRAM ann 64GB RAM, 15-20 t/s on 4080
- Model weight inferencing
Source: r/LocalLLaMA | 2026-10-10