Local Ai
Splash on a 40-core M5 Max: +20% decode by tuning the kernels for your own chip
FYI the engine's default kernel rules were measured on smaller chips (16/20-core M5s and a 32-core M4 Max), so a 40-core M5 Max runs guesses. Splash's repo includes a developer tool, “make tune-kernel
FYI the engine's default kernel rules were measured on smaller chips (16/20-core M5s and a 32-core M4 Max), so a 40-core M5 Max runs guesses. Splash's repo includes a developer tool, “make tune-kernels” that tests every available way of running each quantised matrix-multiply on your hardware. On my machine it found that the "split-K" layouts (each input row split four ways, with the partial sums combined at the end) are much faster for the 8-row step that checks draft tokens. Written up for Inco (https://github.com/incoai/splash/issues/154) In the meantime try it: build Splash from source (git clone https://github.com/incoai/splash, git checkout 1.0.2, make; needs Xcode 26+ with the Metal toolchain), then run build/engine-tests/tune-kernels build/splash.metallib --confirm on an idle Mac. The --confirm step tells you whether the winners actually speed up the whole forward pass on your chip. Use your model of choice to patch in. Swift Model conversions also available on hugging face here: https://huggingface.co/SiliconSpecies/Swift-1.5-Qwen3.8-27B-Splash submitted by /u/SeveralViolins [link] [comments]
Related
- Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.
- Apple unveils a more powerful Mac mini featuring the all-new M6 and M5 Pro
- Building a zero-dependency C inference engine for BitNet (1.58-bit) - lessons from hitting 36 tok/s on a Xeon CPU
Source: r/LocalLLaMA | 2026-09-25