Model Releases
# Qwen3.8-27B — One Week Later: The r/LocalLLaMA + r/LocalLLM Verdict
Companion to the Qwen 3.8 Release Megathread. Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus inde
Companion to the Qwen 3.8 Release Megathread. Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus independent X benchmarks. Every number is attributed to the poster's stated hardware/runtime/quant. This community contradicts itself on nearly every axis — so this thread keeps the disagreements side-by-side instead of picking a winner for you. TL;DR The consensus pick: a 27B dense multimodal model that genuinely moved the bar for local agentic coding. The strongest claim with controlled evidence behind it isn't benchmarks — it's tool-calling reliability. The default ships at xhigh reasoning and it thinks a lot. Low and medium presets score nearly as well on Artificial Analysis (~43/44 intelligence index, within a few points of the xhigh headline) while cutting thinking tokens ~7–9x (and wall time ~6–7x). Most of you should not be running xhigh. Knowledge recall regressed vs 3.6 — widely reported and best understood as a deliberate agentic-design tradeoff. Trivia nerds: keep Gemma around. Q4_K_M is basically indistinguishable from Q8 on perplexity, but real-world reports split hard below Q6 for complex reasoning. KV cache quantization is one of the most contested settings in the corpus. The "neck and neck with DeepSeek V4 / GPT-5.6 Luna Max" AA headline is real but heavily caveated — see the benchmark credibility section before quoting it at your friends. 1. What it's actually good at Agentic coding (strongest consensus area) "Highest level of agency I've ever seen in a local model" (thread): single 3090, Unsloth Q4_K_S + q8 KV, 150k ctx. From one prompt it pulled the OP's class schedule off a convoluted university website via 80 tool calls, zero human intervention. 1M+ token run (thread): RTX 5060 Ti 16GB, UD-Q3_K_XL, 73k ctx. Full REST API + MCP server for a legacy forum from 3 prompts. Controlled tool-call evidence: in a plain Python tool loop (no framework), one reporter got zero failed calls from 3.8 while Gemma 4 A4B and Qwen3.6 A3B failed often — the same reporter who rates 3.8 below both on raw code quality. Worse judgment, perfect plumbing. Creative / game generation One-shot playable Super Mario clone (Q8, Framework Desktop) — top pushback: "It's in the training data." Galaga 1:1 recreation test (UD-Q8_K_XL, 3×3090 + Tesla P40): "This 'Galaga' clone [from 3.6] ended up pretty much being a space invaders clone instead... Qwen 3.8 thinks a LOT, but it draws out those tiny details and absolutely nails it after the fact." A separate r/LocalLLM user one-shot a playable Galaga-style game at IQ4_XS on dual 4060 Tis, and another built an online multiplayer MOBA overnight with an authoritative server and self-play testing. Ray-traced spheres in BASIC: 3.8 self-iterates to a correct Cook-Torrance ray-tracer; 3.6 needed hand-holding. Comment: "this feels more like 3.6 to 4.6 than 3.6 to 3.8." Vision Works natively (F16 mmproj), including OCR-style reading of a newspaper image at ~1,000 image tokens — but on a 16GB card at 64k ctx + MTP it leaves as little as ~150 MiB VRAM free. Practical advice from the 16GB crowd: keep text-agent and vision profiles separate, or offload the projector (--no-mmproj-offload). Where it struggles Long analytical/document work: "a step backwards" vs 3.6 at default settings — though a legal-domain poster got on-par-with-122B results with MCP + case access. Task-dependent. Complex native coding: one failed C kernel effort (6 hours across 3 sessions) [anecdotal], quant unstated; commenters say Q8 minimum for that tier of work. 2. The thinking-level situation (read this before complaining) xhigh is the shipped default. It is why your context window evaporates. Measured ladder (RTX 5080 Laptop 16GB, llama.cpp 10451, UD-IQ3_XXS, Q8_0 KV + FA + MTP, pelican-SVG task, 3 seeds): Effort Reasoning tokens Wall time Visual score /25 Low 4,418 112 s 21.8 Medium 5,918 127 s 22.5 X-High 39,398 718 s 24.0 That's ~6.4x the wall time for +1.5 points on an eyeball task. But on pass/fail SWE-style tasks, xhigh went 9/12 vs 6–7/12 at lower efforts — the premium scales with whether the task has a verifiable failure. How to change it: --chat-template-kwargs '{"reasoning_effort":"medium"}' (llama.cpp) or the equivalent in LM Studio custom params. The overthinking debate, both sides preserved: - Against: "it will do eight or nine web-search turns and spin its wheels down every rabbit hole" (legal work). One reported loop burned 40k+ characters of reasoning on a trivial subtask. One paper-linked post argues intermediate tokens aren't reasoning at all ("Stop Anthropomorphizing Intermediate Tokens," 538 points). - For: "if the extra thinking produces measurably better results it's actually just the correct amount of thinking." The low/medium AA scores (~43/44) are the strongest counter to "it only wins by overthinking" — though two commenters read that same data in opposite directions. Practical takeaway from the corpus: medium for chat/analysis, xhigh only when there's a verifiable right answer. - The strongest controlled effort data of the week is from X: @superalesha's 67-hour, 40-arm run found xhigh burned 7–11× more reasoning tokens than low for 0–4.7 extra points — and in one head-to-head, low matched xhigh exactly (89.3%) at 1/7.5th the tokens. Also: medium scored below low on every stack (all the damage in HumanEval+ — "that preset overthinks short coding tasks"). His verdict: "low is the rational preset. xhigh is for leaderboard screenshots." That's harsher than the Reddit consensus — weigh both, but it's the biggest sample size anyone published this week. More data points from the week: Medium vs xhigh "actually insane" (223 pts): medium ≈ a couple thousand thinking tokens; xhigh 15–20k minimum, one pacman build hit 40k. But the same thread's best counterpoint: on a bug-finding test, xhigh took 7 min vs medium's 80 s and caught every bug; medium only caught the critical ones. And on a research task xhigh autonomously cloned a repo and read source to verify an answer — neither medium nor 3.6 did. Different thinking levels (287 pts): "Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning" — the level you pick changes speed, not whether it beats last generation. There is no "high" effort — the ladder is low / medium / xhigh(default), and the gap between medium and xhigh is the complaint that keeps generating threads. Commenters note the efforts aren't just prompts: Qwen specifically trained each level's instruction text in during RL. Don't confuse budget with effort (PSA): llama.cpp's web-UI reasoning selector is a hard token cap that truncates mid-thought — it is not Qwen's native effort levels, which actually change how thoroughly the model works. On recent builds use --reasoning-effort medium (or the --chat-template-kwargs form on older ones); anything else silently caps instead of steering. The "well?" trick: interrupt mid-think and type well? — the model concludes "the user is impatient, let me finish quickly" and wraps up faster. Works, but commenters consider it a last resort; the thinking is where the quality lives. Dissenters exist: one medium-vs-xhigh post claiming "1/20th the time for almost the same quality" got pushed back hard — top reply: low/medium left them unimpressed, xhigh is where frontier-tier coding shows up. The honest split: for chat and eyeball tasks medium is ~free; for verifiable correctness xhigh keeps earning its cost. 3. Knowledge regression vs 3.6 — real, and deliberate The dedicated thread: 3.8 fails pocket-trivia questions 3.6 reliably answered, at every quant tried. AA's offline Omniscience benchmark agrees. Community framing: 3.8 is trained to go search instead of recalling, i.e., an agent-first tradeoff. Mitigations posted: RAG/MCP (offline Wikipedia ZIM), or run Gemma 4 31B as a knowledge sidecar. Counter-data point: a separate legal-work thread reports Harvey-benchmark scores on par with Qwen 3.5-122B once MCP + case access are attached (61/75 raw vs 71/75 with a tool backend). The knowledge didn't vanish; it moved into the toolbox. 4. Quants: what holds up The one controlled perplexity sweep (16GB-fitting quants, wikitext-2, RTX 5060 Ti) Quant Size PPL vs Q8 Q8_0 27.0GB 6.956 100% Q4_K_M 17.1GB 6.958 99.97% IQ4_XS 14.6GB 7.013 99.2% UD-Q3_K_XL 12.5GB 7.111 97.8% NVFP4 (Q5K) 14.4GB 7.200 96.6% Poster's call: Q4_K_M is the sweet spot; NVFP4 was the biggest disappointment (same size as IQ4_XS, worse PPL). Pushback worth reading: "PPL degrades less than real world performance… ordering flips near the 4-bit level." The Q4-vs-Q6 war (unresolved) Team Q6/Q8: "q8 dramatically better than q4 for complex reasoning"; one user reports flawless 264k-ctx Q6_K_XL sessions, 2 mistakes per 2M tokens. Team Q4-fine: "I run q4 and can only praise the model… just do not go below q8 KV cache." Nuance: "there are like 5 different Q4s and they are not equal" — NVFP4 ≠ MXFP4 ≠ Q4_0 ≠ UD-Q4_K_XL. Past ~Q5 with dynamic quants, differences get hard to detect. The biggest controlled quant test of the week (X) @superalesha ran a 67-hour benchmark: five full production stacks (FP8 vLLM, NVFP4 W4A16 vLLM, AWQ INT4 vLLM, GGUF Q4_K_M llama.cpp, NInfer — all on RTX 3090s), 40 arms across every reasoning effort, 4,800 tasks / 10,120 requests / 14.5M reasoning tokens, no caps. Results: At xhigh every quant landed between 88.0–90.0% pass@1 — AWQ INT4 90.0%, NVFP4/GGUF-Q4_K_M 89.3%, FP8 baseline 88.7%, NInfer 88.0%. The 4-bit quants scored above FP8; McNemar says statistical tie (first vs last = 3 tasks out of 150). "The gap between quants is smaller than the gap between reasoning presets." The weirdest number: GGUF Q4_K_M at low effort scored the same 89.3% as xhigh — on 86k reasoning tokens instead of 651k. Across all stacks, xhigh burned 7–11× more tokens than low for 0–4.7 points. The one statistically real gap: NVFP4 with reasoning OFF collapsed on HumanEval+ (13/30 vs FP8's 30/30, p=0.0041). Flip it to low and it's instantly back to 90/90. Never run reasoning off — it costs 8–12 points everywhere. His cheat sheet: max quality = AWQ INT4 xhigh; daily driver = GGUF Q4_K_M low; honesty note: three of his FP8 arms failed his own methodology audit (leftover token caps) and are being rerun. This largely settles the Q4-vs-Q6 war for this model at task-level benchmarks — but note the tension with the PPL sweep above: perplexity says NVFP4 is measurably worse than IQ4_XS; task performance says they tie. Both can be true (PPL measures token-level divergence; tasks measure whether errors get caught). And community reports of Q4 reasoning loops remain real — "passes benchmarks" and "never loops in a 2M-token session" are different requirements. 1-bit: comedy, not compute Unsloth founder in the 1-bit thread: "I would not suggest folks use 1-bit for agentic use cases / tool calls" — divergence hits 92% from BF16 by token 32. General chat survives; agents don't. If you must: presence_penalty = 1.5. KV cache — among the most contested settings in the corpus f16-vs-q8_0 are not equivalents per one AMD tester (f16 held quality past 120k ctx). But 16GB users run q4_0/q4_1 KV happily at 64k–164k all week. Working rule from comments: don't quantize KV unless you must; if you do, aim ≥ q6; word-of-mouth floor is Q4 model + Q8 KV for agent loops. Unsloth Dynamic v3 notes MTP removed from quants below UD-Q2_K_XL and re-uploaded separately (some users still see draft logs in Q5_K_XL — unresolved). Imatrix released; no QAT used. 5. Performance matrix (attributed) Hardware Runtime / setup Context Result RTX PRO 6000 96GB llama.cpp PR #27342 DFlash2, Q4_K_M 262k 153.9 t/s = 2.26× plain; 304.9 t/s = 4.68× with ngram table (coding prompts); ngram −30% on prose 2× RTX 3090 vLLM + AutoRound INT4 + DFlash2 131k 120 narrative / 218 code decode Single RTX 4090 llama.cpp, UD-Q4_K_XL, MTP + Q4 KV (see X benchmarks below) 130k ~60 t/s Single RTX 4090 same + DFlash2 drafter + --parallel 1 (X) 250k 73.7 t/s RTX 5090 32GB NVFP4-MTP-LOW 262k 121 t/s (vs Q6_K collapsing to 16.3 — 7.5×) RTX 5090 32GB vLLM + unsloth NVFP4, fp8 KV, MTP-2 131k 110–112 t/s sustained RTX 5090 32GB llama.cpp 10536 long gen degrades 122 → 69 t/s within one generation (bug filed) RTX 5060 Ti 16GB UD-IQ4_XS + MTP-1, Q4_0 KV 64k 45.6 t/s Strix Halo 128GB Q8_0 + Q8 KV, ROCm, MTP 142k 9–19 t/s, MTP accept 97–99% RX 7900 XTX UD-Q4_K_XL Vulkan, MTP, q4_0 draft-KV 131k 50–60 t/s; -np 1 made a "HUGE" difference Why "~200 tok/s" claims don't reproduce for you: Windows/WDDM costs 10–15% vs Linux; headlines are measured at short contexts; MTP acceptance is workload-dependent (drops on prose, sometimes net-slower); and the fastest figures come from Blackwell-tuned engines (ninfer), not llama.cpp. X/Twitter benchmark highlights @analogalok's full RTX 4090 matrix: UD-Q4_K_XL on latest llama.cpp. FP16 KV tops out at 100k ctx (40.9 t/s); q8 KV reaches 170k; q4_0 KV fits the full 262k native context in 24GB at 40.7 t/s. Native MTP: 59–60 t/s at 80–130k. Includes exact reproduction flags. His follow-up: --parallel 1 + a Q2_K DFlash2 drafter unlocks 250k ctx @ 73.7 t/s (Q4 KV), 150k @ 75 t/s (Q8 KV), or 90k @ 80.6 t/s (FP16 KV) on one 4090 (requires llama.cpp PR #27342). NVIDIA forums: DGX Spark face-off, SGLang+DFlash2 vs vLLM+MTP, greedy vs official thinking sampler — DFlash2 won. 6. Failure modes & bugs (reproducible ones) Tool-call failures are usually your tool list, not the model. Best controlled experiment in the corpus: 8 undescribed tools → 0/6 successes; the same tool alone → 15/15; 13 described tools mid-list → 0/5, moved to end → 3/3. Give every tool a description, put critical tools last, don't put examples in descriptions. Every framework failure report (Opencode/Pi/Claude Code) has a plain-loop counterexample in the same threads. Hermes harness specifically: constant tool-call failures on vLLM; "perfect, no issues" on llama.cpp --jinja + q8_0 KV at 256k. Template/parser alignment issue, not weights. Hallucinated user instructions during thinking (reproduced on 2 machines, Pi harness): the model imagines an impatient user and once reverted a commit after imagining a French objection. Community fix: the froggeric fixed chat template (see section 7) eliminates the stock-template tool-call/recovery bugs. temp=1.0 garbage output: thinking falls apart into single-character spam within 10–20k tokens across llama.cpp/vLLM, INT4 through BF16. Diagnosis: sampler, not quant. Fixes: temp 0.1, or split sampling (0.8 main / 0.2 post-thinking). Counter-report: temp 0 caused a 70k-token loop instead. No universal setting exists — tune per task. Decode degradation: 122 → 69 t/s within one generation on 5090 llama.cpp; vLLM/ninfer hold >100. Bug filed upstream. Long-context quality drop: an NVFP4+vLLM eval on B200 scored only ~37% correct in its longest context bucket [single report]; separately, a commenter running official BF16/FP8 via the published vLLM recipe reports agents degrading past ~20k tokens and structured outputs breaking past 20k [single report]. Counterpoint: an f16-KV user on UD-Q4_K_XL (ROCm) says their setup held quality past 120k ctx. Config-dependent; verify on yours. Q8 anomaly reports (Unsloth UD_Q8_K_XL offload/CPU pegging): weak evidence, disputed; most Q8 users report zero issues. Reasoning loops at aggressive quants: 40k characters looping on "angry birds" at Q4-with-QKV-quant, including self-aware "I'm stuck in a loop" narration. Never-seen-it-at-Q6 claims abound. 7. The chat-template situation (read before debugging anything) The official Qwen 3.8 Jinja template shipped with real bugs, and the community shipped fixes within 48 hours: Official template issues: enable_thinking=false crashes; multi-turn history gets poisoned with blank hink tags; tool calls crash when your client sends arguments as JSON strings (the standard OpenAI format); mid-dialogue system messages get dropped, wedging agent loops. froggeric/Qwen-Fixed-Chat-Templates (HF, thread, 334 pts) is the consensus drop-in replacement: safe medium default (kills the burn-20k-tokens-then-return-empty xhigh bug), thinking toggle restored, JSON-string tool-call crash fixed, inline effort steering via <|think_low|> / <|think_medium|> / <|think_xhigh|>, and chronological thought preservation for clean KV prefix caching. Actively maintained — v22.1 as of Aug 21. Format-fidelity alternative: a second template stays closer to the exact official prompt format on the theory that deviations subtly degrade quality even when they look fine manually. Pick it if you're benchmarking; pick froggeric for daily driving. Upstream note: llama.cpp merged reasoning_effort forwarding on Aug 14 — recent builds pass reasoning_effort to any template correctly. That fixes the plumbing, not the official template's own bugs. A fixed template is still recommended. 8. Benchmarks: believe selectively Artificial Analysis: headline posts put 3.8-27B neck-and-neck with DeepSeek V4 and GPT-5.6 Luna Max. Low/medium presets score ~43/44 — the key evidence the gains aren't pure overthinking. Agentic index: medium = xhigh − 1 point. The pushback ("A meaningless benchmark", 106 points): the index ranks this 27B above DSV4 Pro, Kimi 2.7 Code, Opus 4.6 and Sonnet 5 — "whatever 'Intelligence' means to AA... is definitely not the same definition we should be using here." Defenders: it's an aggregate skewed toward agentic/science/coding; read the methodology and pick sub-benchmarks for your use case. LiveBench gets respect for monthly task refreshes. Best independent test found: AIME 2026, exact-match, temp 0, pass@1 — FP8-xhigh scored 29/30 (96.7%), tying Opus 4.6 and DeepSeek V4 Pro in the poster's table, vs 94.1% for Qwen3.6-27B. Caveats: single run, problem 7 exhausted the token budget in both precisions (empty, not wrong). Production blind A/B (thousands of tasks): 3.8 wasn't worse at doing the thing — it was worse at knowing when not to do the thing (+50% noise output). Honest calibration: "Opus-level" is real at some tasks, with the right quant and harness. The thread titled "Qwen 3.8 isn't Opus 4.6 level. Let's not be silly." failed at Q6 in VS Code — commenters blamed the editor and the quant, but the burden of proof stays on the demo. 9. Ecosystem: what shipped this week DFlash2 (llama.cpp PR #27342, still in review): 2.26×–4.68× on real coding prompts, +2.7GB VRAM. N-max 5 beats the recommended 7; --spec-draft-p-min silently does nothing; stacking ngram-mod hurt (opposite of DFlash1 on 3.6). ninfer: Blackwell/5090-tuned engine; 120–160 t/s quants; 480 t/s at 4-way concurrency. Likely source of the unreproducible speed screenshots. AutoRound INT4 / AWQ-INT4 GGUFs for vLLM serving. KVarN 4/2-bit KV ported to vLLM 0.27.1 — 262k fits small cards, needle-test passes at 240k, ~20% slower decode. Uncensored/abliterated variants shipped fast: Huihui-ai ablit, an "Uncensored Aggressive" release bundling K_P quants + HauhauCS FastMTP (up to 3.02× TG claimed), and FP8 abliteration reporting refusal rates dropping to 0–6% — with the community counterpoint that the same tables show 30–50% caveat-rate degradation next to those numbers. Quality varies wildly; check benchmark deltas before switching. What's coming 35B-A3B spotted in ms-swift commits (Aug 15). 16GB-card owners are hyped; early numbers suggest ~27–40 t/s on hardware where the dense 27B crawls. A new midsize open-weight model "next week (hopefully)" per Qwen's community manager — no early access this cycle. Speculation centers on ~80B with vision. The flagship Qwen3.8-2.4T-A95B got day-0 vLLM support with open weights announced at launch; it barely appears in this week's local-community threads beyond speed speculation (a 2.4T open-weight Call of Duty clone demo made rounds). Local discussion is overwhelmingly about the 27B. Report template (steal this) So your numbers mean something to the next reader: Runtime/version: Hardware: Model file + quant: KV cache: Speculative (MTP/DFlash2/ngram): Reasoning effort: Sampling: Context size: Prefill tok/s: Decode tok/s: Task used: Compared against: Observed result: Megathread compiled Aug 22, 2026 from r/LocalLLaMA and r/LocalLLM (Aug 15–22) plus public X benchmark threads. All performance figures belong to the hardware/runtime that produced them — the corpus contradicts itself on nearly every axis, and in most cases you can name the variable that explains the split. submitted by /u/Jonathan_Rivera [link] [comments]
Related
- [[megathread-qwen-38-27b-release-day|[Megathread] Qwen 3.8 27B Release Day]]
- Tested in Coding: Q8_K_XL Qwen3.8 27B vs BF16 Qwen3.6 27B
- Qwen 3.8 27b - PI AGENT vs OPENCODE
- Fastest qwen 3.8 27b for AMD gpu?
Source: r/LocalLLaMA | 2026-08-23