Running Vision Qwen 3.8 27B on a 16GB Card, the config (45tks).

I am just sharing my config for Qwen 3.8 27b that fits on a 5060TI, what is cool about this is that you can even get vision! and a 85K context (I have 1.5gb of headroom for more context or a better quant) Model: IQ3_XXS-mtp from https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF Using beellama https://github.com/Anbeeld/beellama.cpp Config used: [*] model = ..llm-modelsQwen3.8-27B-GSQ-RCO-IQ3_XXS-mtp.gguf mmproj = ..llm-modelsmmproj-Qwen3.8-27B-BF16.gguf image-min-tokens = 256 gpu-layers = 99 ctx-size = 85000 no-host = true direct-io = true threads = 8 batch-size = 2048 ubatch-size = 512 fit = off ctx-checkpoints = 0 spec-type = draft-mtp spec-draft-n-max = 2 cache-type-k = kvarn4 cache-type-v = kvarn4 kv-tail-tokens = 256 I managed to get 45tks on decode and around 300 on prefill Yes it is using kvarn4, but it is not that bad, check: https://anbeeld.com/articles/kvarn-kv-cache-implementation-and-benchmarks I also know that you could move the mmproj to cpu to to gain more vram. Would love to hear other configurations to find the sweetspot for 16GB vram cards! submitted by /u/FerLuisxd [link] [comments]

Related

• qwen38-27b-rtx3090 (https://github.com/syv-ai/qwen38-27b-rtx3090) is extremely good with deepseek harness.
• 2 x 5070ti Qwen 27B full config / stats
• After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)
• 1x32GB V100 vs 2x16GB V100 vs 5060ti 16GB for QWEN 3.8

Source: r/LocalLLaMA | 2026-09-10