Model Releases
Qwen 3.6 27B Q5 on 3x2080ti: 55tps with llama.cpp. Can I squeeze out more?
CPU: Threadripper 3970X RAM: 128GB DDR4 GPUs: 3x2080ti 11GB The current best parameters to run it: llama-server --model Qwen3.6-27B-Q5_K_S.gguf --n-gpu-layers 999 --split-mode tensor --flash-attn on -
CPU: Threadripper 3970X RAM: 128GB DDR4 GPUs: 3x2080ti 11GB The current best parameters to run it: llama-server --model Qwen3.6-27B-Q5_K_S.gguf --n-gpu-layers 999 --split-mode tensor --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --ctx-size 16384 --batch-size 2048 --ubatch-size 1024 --threads 4 --spec-type draft-mtp --spec-draft-n-max 3 --no-mmap submitted by /u/AccountGotLocked69 [link] [comments]
Related
- Pipeline parallelism in llama.cpp may be wasting your VRAM
- 80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP
- Deepseek V4 flash - Hy3 or is Qwen3.6 27B still the most solid for agentic/coding?
Source: r/LocalLLaMA | 2026-08-01