Model Releases
b11029
vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 1024 experts (#28501) vulkan: raise the hoisted row-id limit for mul_mat_id to 512 experts The expert-count shader (count_experts.comp
vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 1024 experts (#28501) vulkan: raise the hoisted row-id limit for mul_mat_id to 512 experts The expert-count shader (count_experts.comp) sizes its shared arrays with BLOCK_SIZE, which is 256. Because of that, row-id hoisting is switched off for any model with more than 256 experts, and every mul_mat_id workgroup has to rescan the whole ids tensor on its own. Qwen3.8-Flash-Next has 512 experts and was quietly running on that slow path. This change sizes the arrays with a separate MAX_EXPERTS constant (512), clears them in a loop instead of one entry per thread, and raises the matching limit on the host side. On Strix Halo at batch 2048 the expert matmuls drop from 12.5 to 9.5 ms (iq3_s) and from 14.0 to 7.5 ms (iq4_nl) per op, and prompt processing gets about 19 % faster at 8k tokens. test-backend-ops MUL_MAT_ID passes (891/891) with new 512-expert test cases. Assisted-by: Claude Fable 5.1 vulkan: raise the hoisted row-id limit for mul_mat_id to 1024 experts Follow-up to review feedback: 1024 matches LLAMA_MAX_EXPERTS instead of stopping at 512. The three shared arrays in count_experts.comp grow to 3 * 1024 * 4 = 12 KiB, which fits the 16 KiB that Vulkan guarantees for maxComputeSharedMemorySize. Adds mul_mat_id test cases at 1024 experts alongside the existing 512 ones. test-backend-ops MUL_MAT_ID passes on Vulkan (RADV, Strix Halo, Radeon 8060S): 889/889. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48388682 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13) - CUDA 13.3 libraries Ubuntu arm64 (CUDA 13) - CUDA 13.3 libraries Ubuntu x64 (ROCm 10.0) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.4 DLLs Windows arm64 (CUDA 13) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 10.0) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI
Source: llama.cpp Releases | 2026-09-18