For Strix Halo - Official llama.cpp isn't ideal and how to highest possible throughput

I've been making a lot of comments about optimal setup for Strix Halo (gfx1150) and from my observation, 90% of our community is using offcial llama.cpp for it, which is NOT optimized for Strix Halo at all, official llama.cpp is having extremely hard time to reach 50% hardware theory, wasting the silicon of this device. Here's alternatives that can bring the speed of Strix Halo to a totally different world, I will link to user's sastifaction comment to prove that the result is real: https://github.com/peonist-ai/halogen-flash-server - ~50t/s decode and 1200t/s prefill - 90% theory - Optimized for Strix Halo and Qwen 3.8 Flash Next (Q38FN - The best goto model for Strix Halo) only, if you've heard about Ninfer, this is exactly Ninfer for Strix Halo. https://github.com/myhacsint/llama.cpp/tree/production/strix-halo-qwen4exp-b10685 - Almost 60t/s decode and 600t/s prefill - 80% theory - This is an experiment fork for Q38FN, having very high decode and pretty good prefill https://github.com/halo-box/strix-llama.cpp - Almost 30t/s decode and 800t/s prefill - 75% theory - The latest commit pushed prefill through the roof, this is r/StrixHalo very first fork that has been updating nonstop with a Discord server with a lot of users. Note: Official llama.cpp running Qwen38FN at 2xt/s and 2xxt/s prefill - 50% theory. Hopefully this will be helpful to the Strix Halo users. submitted by /u/feelspeaceman [link] [comments]

Related

• Strix Halo (8060S / gfx1151), Qwen-3.8-27B @ Q8 and Q6 UD v3, up to 256K ctx, llama.cpp, DFlash2, vision, real workloads quality and steady performances, optimized recipes, ...
• Qwen3.8 27B on Strix - the optimized setup
• 2x Strix Halo speed-up with an R9700
• Qwen3.8-27B at 262K context on a Strix Halo + RTX 3090 Ti: 9.5 -> 153 tok/s, and it beats a dual-3090 vLLM box on HumanEval

Source: r/LocalLLaMA | 2026-09-08