strix-llama new branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm kernel at 128k, same outputs quality

I've been tuning llama.cpp on my Strix Halo box (Radeon 8060S, 128 GB) and the numbers are good enough to share. It's a branch called rdna35-perf on top of halo-box/strix-llama.cpp, a llama.cpp fork aimed at Strix Halo: https://github.com/asf0/strix-llama.cpp/tree/rdna35-perf (10 commits, ROCm/HIP only) TL;DR Prompt processing: 21–54% faster than Vulkan, and 8–174% faster than stock ROCm. The lead over ROCm grows with context, because stock ROCm flash attention falls off hard at depth. Generation: ties Vulkan on the dense 27B. On the MoE and the 2B it's 2–3% behind Vulkan at short context and within 1% from 64k up. 5–29% faster than stock ROCm. Quality: outputs stay closer to stock ROCm than Vulkan's do, and perplexity is unchanged. Model PP vs Vulkan (0k → 128k) PP vs stock ROCm (0k → 128k) TG vs Vulkan (0k → 128k) Qwen3.6 35B-A3B (MoE) +54% → +36% +38% → +88% −3% → 0% Qwen3.8 27B (dense) +38% → +25% +8% → +174% 0% → 0% Qwen3.5 2B (dense) +21% → +34% +9% → +87% −3% → 0% Results llama-bench, Q8_0 weights, q8_0 KV cache, flash attention on, tok/s. "Depth" is how many tokens are already in the KV cache when the test runs. Vulkan and ROCm are the same strix-llama.cpp master that rdna35-perf is built on. Qwen3.6 35B-A3B (MoE, ub 4096) Test Depth Vulkan ROCm rdna35-perf vs Vulkan vs ROCm pp4096 0 2625 2935 4045 +54% +38% pp4096 8k 2304 2430 3452 +50% +42% pp4096 16k 2065 1966 3004 +45% +53% pp4096 32k 1716 1460 2388 +39% +64% pp4096 64k 1279 959 1695 +32% +77% pp4096 128k 786 568 1068 +36% +88% tg64 0 58.2 52.3 56.3 −3% +8% tg64 8k 56.1 50.4 54.3 −3% +8% tg64 16k 54.6 48.5 53.2 −3% +10% tg64 32k 51.8 45.3 50.7 −2% +12% tg64 64k 47.1 39.7 46.4 −1% +17% tg64 128k 39.9 32.2 39.9 0% +24% Qwen3.8 27B (dense, ub 2048) Test Depth Vulkan ROCm rdna35-perf vs Vulkan vs ROCm pp2048 0 484 614 666 +38% +8% pp2048 8k 442 542 611 +38% +13% pp2048 16k 421 476 575 +37% +21% pp2048 32k 385 348 513 +33% +47% pp2048 64k 326 204 422 +29% +107% pp2048 128k 250 114 312 +25% +174% tg64 0 8.0 7.6 8.0 0% +5% tg64 8k 7.9 7.4 7.9 0% +7% tg64 16k 7.8 7.2 7.8 0% +9% tg64 32k 7.6 6.8 7.6 0% +12% tg64 64k 7.3 6.2 7.3 0% +18% tg64 128k 6.8 5.2 6.7 0% +29% Qwen3.5 2B (dense, ub 2048) Test Depth Vulkan ROCm rdna35-perf vs Vulkan vs ROCm pp2048 0 6481 7179 7852 +21% +9% pp2048 8k 5618 6253 7158 +27% +14% pp2048 16k 5000 5172 6611 +32% +28% pp2048 32k 4155 3888 5580 +34% +44% pp2048 64k 3220 2597 4277 +33% +65% pp2048 128k 2160 1547 2886 +34% +87% tg64 0 99.3 91.5 96.8 −3% +6% tg64 8k 95.9 88.2 93.2 −3% +6% tg64 16k 92.9 84.5 91.4 −2% +8% tg64 32k 88.3 79.1 87.1 −1% +10% tg64 64k 79.5 69.2 79.2 0% +14% tg64 128k 67.6 56.2 67.7 0% +21% f16 KV gives the same picture: PP 20–56% faster than Vulkan and up to +171% over stock ROCm at 128k, TG within −3% / +2% of Vulkan. Quality rdna35-perf changes kernels, so I checked it doesn't change outputs: KL divergence against stock ROCm on wikitext-2 (6 × 4096-token chunks), plus perplexity at 32k context. Model KLD vs stock ROCm (Vulkan / rdna35-perf) Same top token (Vulkan / rdna35-perf) PPL @ 32k (Vulkan / ROCm / rdna35-perf) Qwen3.6 35B-A3B 0.0038 / 0.0018 97.8% / 98.5% 6.028 / 6.023 / 6.024 Qwen3.8 27B 0.0066 / 0.0013 98.9% / 99.7% 6.003 / 6.004 / 6.007 Qwen3.5 2B 0.0008 / 0.0001 98.6% / 99.5% 10.046 / 10.049 / 10.049 rdna35-perf is closer to stock ROCm than Vulkan is, and perplexity is the same across all three builds. What changed Flash attention kernel for RDNA3.5 (fattn-rdna35). This is the biggest win and the reason long-context PP holds up. GGML_FA_RDNA35=0 turns it off if you want to compare. Prefill GEMM tuning for the quantized matmuls. MoE: routing helper, fused reduction + residual, grouped mat-vec for speculative-decoding verify batches. Gated DeltaNet (the linear-attention layers in Qwen3.5/3.6/3.8): fused SiLU-gated output norm and fused decode. Decode: plain mat-vec for rows longer than 4096 (this closed the 27B decode gap to Vulkan), plus decode fusions for the Qwen3.6 MoE. Misc: HIP graphs off by default on RDNA3.5 (slower here), a faster penalties sampler, host-side prefill for Qwen3.5/3.6 with a draft model, and DFlash drafts read target features in place on the GPU. I have only tried those 3 models so far. submitted by /u/asfbrz96 [link] [comments]

Source: r/LocalLLaMA | 2026-10-12