Model Releases
Over 200k context on 16GB VRAM with Qwen 3.8 27B UD-IQ3_XXS
I was using UD-Q3_K_XL until now with more than 140000 context. Quality wise it's very good, very few erroneous tool calls. Then I saw many others here reporting good results with IQ3_XXS, so I gave i
I was using UD-Q3_K_XL until now with more than 140000 context. Quality wise it's very good, very few erroneous tool calls. Then I saw many others here reporting good results with IQ3_XXS, so I gave it a try. The downside is prompt processing speed went down from 700-800 tk/s to 400 tk/s. Quality difference is yet to be tested. KV cache were both quantized to q5_1 (llama.CPP compiled with DGGML_CUDA_FA_ALL_QUANTS=ON) Served without MTP and mmproj. My setup is a measly laptop with TB4 and Aorus 5060ti AI Box eGPU. Windows 11, cuz Nvidia. Apologies for any mistake in the post. submitted by /u/abskvrm [link] [comments]
Related
- After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)
- Real local agentic coding on a 12GB VRAM budget.
- 16 GB VRAM purgatory discussion thread
Source: r/LocalLLaMA | 2026-08-27