Nanbeige4.2-3B: I'm not impressed

I've tested Nanbeige-4.2-3B. On paper, the benchmarks promise it blows away Qwen3.5-9B and Gemma4-12B. My goal was to have something very light and fast to replace Qwen3.6-35B (or finetunes thereof) for simple and straightforward coding tasks. In the past I tried downgrading Qwen3.5-9B and it was not good enough to be considered. The model is currently broken in llamacpp master - this PR fixes it: https://github.com/ggml-org/llama.cpp/pull/26324 After fixing its issues, I played around with it and must say I'm not impressed. To begin with, it's a looped model: all layers are traversed twice. This means that, at a theoretical baseline, it has the speed and context size of a 6B model. It's nice to be able to run the weights at a Q6 quant and barely notice the size difference from Q4, but you will have to compensate by using a very bad KV cache quant, because the context is enormous for the size. 128k of kvarn3 t2048 context, which I must point out is both very tight and at the edge of the cliff of what is usable without extreme degradation, costs 5.2GB. That's ginormous for a model this size. 256k kvarn5 won't fit on 16GB VRAM after you factor in weights and desktop. The model uses the same "hack" to get good benchmark results that Laguna-S-2.1 uses: at [max] thinking level, where it is benchmarked, it thinks and thinks and thinks and just does not stop. This means that, besides being atrociously slow (wall time per task), it burns through its context budget VERY fast even for simple tasks. I gave it two very straightforward, uncomplicated brownfield maintenance tasks in a project with a robust AGENTS.md and skills. It flunked both. The only good thing I have to say is that tool calling is rock solid. After the llamacpp PR above, it never fails a single tool call. Is it actually better than Qwen3.5 9B? Hard to say: I've only had bad experiences with that too and I have a hard time telling apart models that consistently fail at the simplest tasks. Worth noting that Nanbeige has exactly the same size in memory (at 128k) and same speed. time-per-task, Qwen3.6-35B-A3B with experts spilled to host memory is vastly faster and actually produces correct outputs. Want something small? Not a good model (tiny on disk, enormous in VRAM). Want something fast? Also no, particularly when you measure time-per-task instead of tok/s. Want something precise and reliable for the very easy stuff? Also no. submitted by /u/crusaderky [link] [comments]

[[paper-gpqa-mmlu-pro-and-mmmu-pro-were-audited-for-broken-que|[PAPER] GPQA, MMLU-Pro, and MMMU-Pro were audited for broken questions, and up to 12% of them had to be removed. New drop in clean versions released]]
Deepseek V4 flash - Hy3 or is Qwen3.6 27B still the most solid for agentic/coding?
Cactus Hybrid: We taught Gemma 4 to know when it's wrong
Has anyone compared pre-training, SFT/LoRA and reinforcement post-training on Qwen3.6-27B?

Source: r/LocalLLaMA | 2026-07-30

Related