Model Releases
Qwen 3.8 27B SlopCodeBench results
Howdy, I'm back again - running my favorite benchmark (it's still unsaturated for the time being so might as well!) previous runs a b https://github.com/michaelasper/benchmarks/blob/main/qwen3.8-27b-p
Howdy, I'm back again - running my favorite benchmark (it's still unsaturated for the time being so might as well!) previous runs a b https://github.com/michaelasper/benchmarks/blob/main/qwen3.8-27b-pi-on-slop-code-bench.md I ran this via OpenRouter because my mac would cry running 9 problems It did pretty poorly on the strict checkpoints which means you probably don't want Qwen managing the codebase by itself as it'll grow unwieldy and disorganized, but did fairly well on the core checkpoints so it can solve issues with a copilot and clear direction AI;DR here are the direct results HumanLayer Opus 5 Benchmark Subset (3 Problems, 17 Checkpoints) Qwen scored 3/17 (17.6%) strict. Reported System Strict Score DeepSeek V4 Flash 0731 · pi (run B) 5/17 (29.4%) Opus 5 · Claude Code 4/17 (23.5%) Qwen3.8-27B · pi 3/17 (17.6%) DeepSeek V4 Flash · OpenCode 3/17 (17.6%) Opus 4.8 · Claude Code 1/17 (5.9%) Sonnet 5 · Claude Code 1/17 (5.9%) HumanLayer Fable, Sol, and Kimi Benchmark Subset (6 Problems, 30 Checkpoints) Qwen scored 4/30 (13.3%) strict. Reported System Strict Score Fable 5 · Claude Code 10/30 (33.3%) GPT-5.6 Sol · Codex 10/30 (33.3%) Kimi K3 · Modal / OpenCode 8/30 (26.7%) Kimi K3 · Baseten / OpenCode 7/30 (23.3%) Qwen3.8-27B · pi 4/30 (13.3%) submitted by /u/corruptbytes [link] [comments]
Source: r/LocalLLaMA | 2026-08-19