Qwen3.8 27B Q2 vs Q3 vs Qwen3.6 35B-A3B MoE on 12GB VRAM

Did a quick local test because I wanted to see what is actually usable on my 12GB laptop GPU. I tested the newer Qwen3.8 27B dense files at Q2 and Q3, then compared them against Qwen3.6 35B-A3B MoE. Hardware: RTX 5070 Ti Laptop, 12GB VRAM Backend: llama.cpp CUDA Settings: 4k context, q8 KV, --fit on, no MTP Command shape: llama-cli -c 4096 -fa on -ctk q8_0 -ctv q8_0 --fit on -fitt 768 --temp 0 --single-turn Models: Qwen3.8-27B-UD-Q2_K_XL Qwen3.8-27B-UD-Q3_K_XL Qwen3.6-35B-A3B-Q4_K_M ​ Main result visually: generation speed on the sanity prompt https://preview.redd.it/io9im15fgsjh1.png?width=1537&format=png&auto=webp&s=c3caca8369e22bc45838e7fd1fdf7e1831e64fdf Test 1: tiny sanity prompt Prompt: Answer only as: 1) ... 2) ... 3) ... 4) ... 5) ... 6) ... If a shirt costs 20 after a 20% discount, original price? In the word strawberry, how many r letters? Which is heavier: 1 kg steel or 1 kg feathers? A bat and ball cost 1.10 total. Bat costs 1 more than ball. Ball price? If all bloops are razzies and all razzies are lazzies, are all bloops lazzies? What is 17*24? Model Prompt t/s Gen t/s Result Qwen3.8-27B Q2 dense 412.1 35.9 5/6 Qwen3.8-27B Q3 dense 95.6 7.5 6/6 Qwen3.6-35B-A3B Q4 MoE 155.8 59.0 6/6 Qwen3.8 Q2 missed the bat/ball question and answered 0.10. Correct answer is $0.05. Qwen3.8 Q3 got all 6 right, but generation was only 7.5 t/s. MoE got all 6 right and generated at 59 t/s. Test 2: short normal answer Prompt: Answer in one short paragraph: for a 12GB VRAM laptop GPU, would you prefer a dense 27B 2-bit/3-bit model or a 35B-A3B MoE? Mention speed, quality, and offload. Model Prompt t/s Gen t/s Qwen3.8-27B Q2 dense 309.8 35.5 Qwen3.8-27B Q3 dense 95.0 8.1 Qwen3.6-35B-A3B Q4 MoE 81.8 52.6 Test 3: tiny coding task Prompt: Output only JavaScript code. Write a function winner(board) where board is an array of 9 strings: "X", "O", or "". Return "X" or "O" if that player has three in a row, "Draw" if full with no winner, otherwise "Pending". No explanation. Model Prompt t/s Gen t/s Result Qwen3.8-27B Q2 dense 277.5 34.9 correct Qwen3.8-27B Q3 dense 122.9 9.1 correct Qwen3.6-35B-A3B Q4 MoE 97.3 51.9 correct I also tried a more open-ended “make a full single-file HTML tic-tac-toe game” prompt, but that was honestly a bad benchmark because all three started styling too much and hit the token cap before finishing. The compact JS function was a cleaner test. My rough takeaway Qwen3.8 Q2 is actually usable speed-wise, but it already showed a quality hit on a tiny sanity test. Qwen3.8 Q3 keeps the sanity-test quality, but it is painfully slow for interactive use on my setup. The MoE is still the easy winner for me. It was faster than both dense runs at generation, passed the sanity test, and did fine on the small coding prompt. So if the goal is “can I run Qwen3.8 27B on 12GB VRAM,” then yeah, Q2 can move. But if the goal is “what would I actually use for local chat/coding,” I’d still pick Qwen3.6 35B-A3B MoE right now. Not claiming this is a real eval. Just a small local sanity + speed test. If anyone has better coding/agent-loop prompts, drop them and I’ll rerun all three. submitted by /u/CoffeeToCode99 [link] [comments]

Related

• Qwen 35B-A3B MoE vs 27B dense in local coding tests: ~4× faster, much smaller quality gap than I expected
• club-5060ti refresh: tested RTX 5060 Ti presets, a proper high-context harness, and Qwen3.8 27B
• I tried running a 1.56TB MoE model on a 6GB RTX 4050 Laptop, Here’s the result
• I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8

Source: r/LocalLLaMA | 2026-08-16