Model Releases
b10908
metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 Generalize the row split from #28086 to
metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 Generalize the row split from #28086 to the six other kernels that use the same lane-to-block mapping: iq1_s, iq1_m, iq2_xxs, iq2_xs, iq2_s and iq3_s. Each of them assigns one 32-element chunk per thread, so when a row has fewer than 32 chunks the rest of the simdgroup is idle. When nb32 < 32 and nb32 divides 32, 32/nb32 threads now share each chunk and each takes a slice of the rows, reusing the FC_mul_mv_split function constant and the dispatch wrapper introduced for iq3_xxs. The plain path is untouched: wide matrices keep one thread per chunk and N_R0_ = 4. Only the split path uses N_R0__SPLIT = 8. The K-quants have the same idle-thread issue but a different lane mapping, so they are left for a separate change. metal : offset the src0 row pointer once in the iq mul_mv kernels q2, dh, sc, qh and signs are all derived from xr, so the row slice offset only has to be applied to xr. metal : fold iq mul_mv row split into offset0 Compute row0 and row1 before initializing the source pointers and apply the row slice directly to offset0. This keeps x and its derived pointers on the existing path while applying the split row offset once. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46839299 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (ROCm 10.0) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.3 DLLs Windows arm64 (CUDA 13) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 10.0) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI
Related
Source: llama.cpp Releases | 2026-09-11