Model Releases
Multi-hour llama.cpp optimization experiments on Qwen MoE models, patches, benchmarks, and reproduction guides
I had Astra run a multi-hour investigation into speeding up local MoE inference while keeping the model weights and quantization unchanged. The experiments covered Qwen3.6-35B-A3B and Qwen3.8 Flash-Ne
I had Astra run a multi-hour investigation into speeding up local MoE inference while keeping the model weights and quantization unchanged. The experiments covered Qwen3.6-35B-A3B and Qwen3.8 Flash-Next. Some workloads showed substantial gains, particularly prompt processing and source-code editing. There were also regressions and tradeoffs, which are documented alongside the results. Hardware: RTX 4080, 16 GB VRAM Ryzen 9 5900X 64 GB DDR4 RAM I’ve shared the source patches, benchmark summaries, and reproduction guides here: https://github.com/Abzolute1/llama-cpp-optimization-lab The goal was better performance without sacrificing output quality. The tests include correctness checks, but they don’t establish unchanged quality across every task. You can read through the findings yourself or give the repo to your coding agent. The portable clean build route still needs end-to-end validation, so I’d welcome independent reproduction,especially on different hardware. submitted by /u/Aelexi93 [link] [comments]
Related
- Strix Halo (8060S / gfx1151), Qwen-3.8-27B @ Q8 and Q6 UD v3, up to 256K ctx, llama.cpp, DFlash2, vision, real workloads quality and steady performances, optimized recipes, ...
- Qwen 35B-A3B MoE vs 27B dense in local coding tests: ~4× faster, much smaller quality gap than I expected
- 10%+ performance improvement on MoE ssd-streaming with expert-lookahead
Source: r/LocalLLaMA | 2026-09-18