v0.4.1

Overview llama.cpp 0.4.1 adds Maple 20B-A1B, Tencent Hy 4, and Spark2.5 support, improves JSON schema handling, chat parsing, logging, and server child-process management, and updates ggml to v0.24.0. API changes Changed llama_sampler_chain_n() to return int32_t instead of int (#28631). Added server_subproc and waiter for router child-process monitoring in server-common.h (#28555). New models Added Maple 20B-A1B ternary MoE architecture (CPU) (#27000). Added Tencent Hy 4 (hy_v4) preview architecture support (#28127). Added Spark2.5 support (#27868). Core changes Added Kimi-K3 recurrent-state rollback support (#28466). Fixed MTP context KV cache allocation for DeepSeek2, GLM-MoE, and related models (#28630). Fixed GDN normalization from max to rsqrt for affected Qwen/Kimi/GLM models (#28068). Fixed Granite parameter counts and Granite3 MoE unknown parameter counts (#28643, #28632). Fixed MIMO2 SWA pattern loading and incorrect get_key_or_arr uses (#28865, #28868). Added --fuse-qkv conversion flag to fuse Q/K/V tensors during HF-to-GGUF conversion (#22780). Wrote explicit recurrent_layers metadata for Qwen3-Next/Qwen3.5 conversion (#28208). Refactored JSON schema handling with a common_schema internal representation (#28736). Split specialized chat parsers into common/parsers (#27764). Added structured JSONL logging via --log-jsonl and LOG_JSON (#28437, #28586). Removed deprecated --mmap/--mlock/--direct-io args in favor of --load-mode (#28334). Made mmproj and draft devices default to the global --device selection (#28390). Disabled lazy tensor loading by default on iGPUs (#28326). Fixed speculative decoding after multimodal input and DFlash mtmd chunk decode (#28715, #28587). Fixed Jinja dot-property integer literals and null in membership (#28817, #28620). Improved grammar insert/move handling and max repetition threshold (#26885, #28469). Improved qwen3-coder complex-type parsing and typed content detection (#28742, #28511). Added PCH and unity build support to improve build times (#28091). Updated cpp-httplib to 0.56.0 (#28787). Multi-modality changes Propagated video hash to bitmap IDs ({hash}+{frame}) for cache identification (#28601). Fixed DFlash speculative decoding failures on mtmd chunks (#28587). Server changes Refactored server child-process handling into a single monitor thread (#28555). Fixed LRU hang on multiple requests for the same model (#28539). Allow model downloads when --models-max is already reached (#28530). Fixed context checkpoint eviction on prompts shorter than checkpoint_min_step (#28302). Fixed speculative decoding after image input (#28715). Frame router child state commands as whole lines (#28747). Enabled --reasoning-preserve by default (#28437). UI changes Improved chat message rendering performance with lazy mounting and compositor-friendly animations (#28460). Added UI asset cache for faster builds (#28802). Embedded UI assets directly with CMake, removing the build-time helper (#28445). Fixed MCP image attachments not displayed in tool blocks (#28089). Export conversations from the database instead of the cached store (#27432). ggml changes Bumped ggml from v0.23.0 to v0.24.0 (release). Version 0.24.0 focuses on expanding backend coverage and robustness, with a new precision-control API, major Vulkan/SYCL/Hexagon/OpenCL work, and numerous correctness and performance fixes across CPU, CUDA, Metal, and other backends. Assets Nightly build: b10964 More info Releases and versioning of ggml-org projects Changelog since v0.4.0 b29c606 llama.cpp : bump version to 0.4.1 (#28900) d9e03f1 sync : ggml eeea731 ggml : bump version to 0.24.0 (ggml/1627) bbdd9f2 tests : add fusion baseline README and broaden fusion CI triggers (#28893) 97e4ca7 models : fix incorrect uses of get_key_or_arr (#28868) 1aca1f9 models : fix mimo2 swa pattern load (#28865) be2c6d7 tests(s390x): add non-vxe build to tests (#28776) 3d10bcd llama: add Maple 20B-A1B ternary MoE architecture (CPU) (#27000) 21f6b0d sycl: rfc: Use radix select for top_k (#28670) 2f53959 ggml-cpu : disable PCH and fix CACHE_LINE_SIZE ambiguity to fix heap corruption (#28882) 89fe242 ci : trigger self-hosted CI on changes to ci/run.sh (#28859) 15d8f2d ci : remove gg_sum summary logic (#28857) 661643e sycl : fix oneDNN scratchpad breaking the pool free order (#28704) 093a2f8 common : move llama_n_rs_seq to before llama_decode (#28749) ad6c668 ggml-cuda: fallback to F32 on device without BF16 hardware acceleration (#28846) 7a16a6c grammar : coalesce find + insert into a single insert and adjust move/copy mechanics (#26885) 5f436dd tests : exclude HY_V4 from WebGPU test-llama-archs tests (#28855) e49d2c2 models : guard the expert FFN size fallback in nemotron-h against a zero divisor (#28779) 6978052 ggml-cpu(s390x): guard VXE-only repack helpers (#28775) 243a308 tests : fix typo in test-quant-type-selection for nemotron 3 nano (#28835) b6b003d sycl : Fix get mem error (#28227) c95f8e4 ci : run editorconfig and code-style checks on ubuntu-slim (#28854) bc52a12 pi : prefer PI_MODEL_NAME env var for model disclosure (#28853) 4a89937 tests : reduce FA test sizes (#28842) 37b3a9e ci : remove leftover command (#28839) 002a12a ci : cap test-backend-ops parallel jobs at 2 and add a 3600s timeout (#28833) f1e44dc vulkan: workaround NV queuesubmit driver bug (#28830) 56b9eb2 opencl: apply the noshuffle row-alignment rule to q4_K, q5_K and q8_0, not just q6_K (#28575) 790cf51 chat : improve parsing of complex types in qwen3-coder (#28742) 8e33095 common: add LOG_JSON macro to log structured data (#28586) acecd56 common : implement common_schema internal representation for JSON schemas (#28736) ae9afff jinja : support dot property integer literals (#28817) 737e098 cmake: leave the timestamp out of precompiled headers on clang (#28816) 3057bb6 ui : add cache (#28802) 56381e4 server : allow model downloads at model limit fix issue #26809 (#28530) c8edceb ggml-cuda: hip add specific config table for AMD GCN (#27841) e192abb server : add missing headers (#28795) 718f7b4 vendor : update cpp-httplib to 0.56.0 (#28787) 2a3005c syscl : Handle (fail gracefully) unsupported tq1_0 quants (#28681) f3a33df rpc : fix linking when compiling with BUILD_SHARED_LIBS=OFF (#28492) c069aa7 server: frame the router child state command as a whole line (#28747) 8a56aed opencl: fix several bugs where the backend aborts (#27630) 07fc977 opencl: add bin kernel kernel_gemm_noshuffle_q4_k_f32_32b_trans_ila_a8_bin (#28677) 3f5e94d webgpu: align tensor bindings to the type block size (#28382) eafe15a hexagon: support for multi-device model split (aka row-split) (#28589) d3146f2 ggml-webgpu: Update to a recent version of Dawn (#28683) 82d6bb2 server: refactor subproc handling (#28555) 8ea2902 cmake : skip PCH for llama-server PCH when using MSVC (#28763) b78a39a ci : run test-backend-ops as a dedicated ci/run.sh test (#28740) 982937a tests: extend test-quantize-fns to test nrc=2 (i8mm) kernels (#16234) 8172e65 tests: tolerate a shared pool abort in test_completion_unified (#28759) 43f3dda ggml: skip 0-sized ids tensor when offloading selected experts (#28739) 5bda51b metal : skip the empty half of the mul_mm_id token tile (#28301) 3bcfeb7 cmake : add PCH and unity build to improve build times (#28091) 1dfe94e common : fix typo in speculative.cpp comment [no ci] (#28750) a2878d3 metal : single-source fusion table + fusion debug rework (#28164) aac8102 metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) 5cdd3d1 model : fix MTP context kv cache allocation for deepseek2, glm4moe, … (#28630) b0dcb81 server: fix speculation after an image (#28715) 16378d9 CUDA/HIP: Flash Attention tuning (gfx1201) (#28102) 451b89b ci : key cache to sanitizer matrix (#28708) 481c65f vulkan: fix data race and OOB access in argsort(large) (#28705) df03399 opencl: add A8 Q4_0 mm binary kernel support (#28268) 28ff095 vulkan: use CPU writes in ggml_backend_vk_cpy_tensor_async if the context is idle (#28618) 50182a5 vulkan: use add_alloc_dep to enable topk_moe fusion for prefill (#28422) 6788edb vulkan: small M matrix optimizations for qwen (#28457) 52d4268 ci : add self-hosted-gpu-cuda and server-sanitize to hf-jobs (#28693) 18c17b4 ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (#28687) fa67698 spec: fix failed to decode mtmd chunk with DFlash (#28587) 41fc758 scripts : use sed instead of grep for version parsing [no ci] (#28700) d344123 models: clean up some dead switch branches in old models (#28669) c32d1da tests : increase tolerance for Add fusion tests (#28691) e5a8d43 tests : drop SYCL special-casing in test-backend-ops.cpp (#28688) 3ff67eb vulkan: fall back to shared-memory reduction for dmmv on PowerVR (#28341) 8c322d5 convert : expand Nemotron H conversion fix (#28689) 311d421 memory : avoid allocating V cache for indexer (it's not used) (#28330) 72797e8 vulkan : add command-buffer debug labels for GPU profilers (#28101) 4ea6d1b ggml-cpu(s390x): add repack support for q4_0 (#28667) f1b6fbf ggml-cpu(s390x): add Q1_0 vector intrinsic support (#28606) d7e8643 model: fix all granite family parameter counts (#28643) 434ddbb ci: fix sanitizer tests (#28583) 91f6a6c vulkan: use spec constant for matrix matrix multiplication A-type (#25773) 6d9c82e hexagon: rope updates (#28628) 22397c3 vulkan: Convert FILL to distribute workgroups in 2D to avoid exceeding maxComputeWorkGroupCount (#28592) 4b98ab8 py : lower numpy to 2.2.6 (#28654) 9cf3bf2 py : bump numpy to 2.4.6 (#28649) 4850c77 llama : use int32_t for llama_sampler_chain_n return type (#28631) d4abd57 CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#28552) 5a4d0fe CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (#28079) 14a9d09 args: officially deprecate --mmap|mlock|dio (#28334) e2d2c0d model: fix granite3 moe unknown parameter count (#28632) 6de9cdb mtmd: propagate video ID to bitmap (#28601) b31b71f jinja: treat a null left operand of in as a plain lookup (#28620) df750f7 vulkan: add dedicated iq4_xs mat-vec shader (#28426) 1945e09 vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 (#27471) 30b6a75 tests : use less threads for data initialization (#28325) 304665f Add IQ type handling for MoE (#28476) f3f1a8f llama: disable lazy tensor loading by default on iGPUs (#28326) 9113cc1 ggml : fix msvc+clang ggml_vld1q_u32 (#28284) d4389a4 Revert "ggml-cuda : restore prop.integrated on HIP builds (#24233)" (#28604) 5d806aa server : apply checkpoint min-step eviction only when the checkpoint list is full (#28302) 88ada91 metal : fix idle threads in mul_mv_iq3_xxs for ne00 < 1024 (#28086) 415e909 spec: single device drafter should create meta backend wrapper (#28390) 03fa73c ci : disable npm gha cache (#28600) 1744c6b ci : add PYTEST_WORKERS=1 to fix server-self-hosted job (#28603) ca86fb2 llama : add missing headers (#28566) 64e9bce vulkan : fuse UNARY(GELU|SIGMOID|SILU|SOFTPLUS) + MUL (#27220) f014bfe Fix Vulkan-Hpp handle usage on 32-bit targets. (#22892) 895c045 chat : split specialized parsers into common/parsers (#27764) 7d701b5 opencl: properly handle non-contiguous inputs to conv2d (#28503) 5a6caa0 ggml : update ggml_prec specification (#26675) 9dcf84e model : support Kimi-K3 recurrent-state rollback (#28466) 050dde5 hexagon: add RELU and LEAKY_RELU ops (#28585) 67672dc ci : bump ty to 0.0.78 (#28548) f114f91 tests : initialize the L2_NORM batch array (#28553) e71b805 Revert "CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#24546)" (#28551) ccc3646 nix : update deprecated expressions (#28145) c0b1871 webgpu: format the GET_ROWS case block (#28542) 160bd03 server: fix LRU hang on multiple requests same model (#28539) dbeb375 sycl: add a batched L2_NORM kernel (#28222) 7a333e7 vulkan: add DeepSeek-V4 hyper-connection fused ops (DSV4_HC_COMB/PRE/POST) (#26578) 0c96345 CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#24546) 4735997 ggml: add gfx90c HIP support (#26454) d23c47f convert : refactor Hy4-preview conversion - move HC tensor mapping to the global map (#28451) 73ab759 CUDA: branchless Q4_K/Q5_K unpack to speed up mmvq, L2 prefetch on DGX Spark (#26705) 0cae430 vulkan: support type-aligned GET_ROWS (#28253) 1173700 examples : print ggml_version and ggml_commit in test-cmake [no ci] (#28538) 5202104 caps : recheck typed content if template checks for string (#28511) 9a75705 convert : write explicit recurrent_layers for Qwen3-Next / Qwen3.5 (#28208) b74f590 ggml-cuda: fix divergent barrier in f16 flash attention (#27870) 992cb50 ggml: allow backend inputs to not create another split (#28387) 9ac8c40 vulkan: rms_norm fusion opportunities (#28024) 2092353 ci : add container image checking and tagging (wip) (#28394) 8fe90e1 vulkan: add TQ1_0 support (mm, mat-vec, mat-vec-id, dequant, get_rows) (#27765) 465e49b convert : add --fuse-qkv flag to fuse Q/K/V into QKV during HF-to-GGUF conversion (#22780) 5fdfa62 models : fix GDN normalization from max to rsqrt (#28068) 3ad1ba7 [Model] Support for Spark2_5ForCausalLM implementation (#27868) d03efa5 opencl: properly choose weights pack for q4_K, q5_K mul_mat (#28402) 73a43d1 cuda: fixes races in mmid and mmf (#28475) 9e0e220 grammar : fix max repetition threshold (#28469) 0afb805 ui: Improve Chat Messages rendering performance (#28460) 7620399 common: add --log-jsonl (#28437) c457e3b ui : embed assets directly with CMake (#28445) 971595d metal : add remaining fa-vec tunings for M2 Max (#28458) 74a7c89 Github: limit blank issues to maintainers (#28435) 6a1a922 metal : fix memory leak in early return (#28399) 4d91760 sycl : fix test-backend-ops CI break && restore Kronecker product FWHT support (#28016) (#28254) cd8cdf3 sycl: attribute device allocations by site (GGML_SYCL_MEMTRACE) (#27631) 427291b metal : add remaining fa-vec tunings for M3 (#28396) 85d5703 ui : fix MCP image attachments not displayed in tool block (#25789) (#28089) 1548a24 opencl: extend the elementwise and data‐movement op coverage (#27633) 4acf4a4 opencl: add Adreno xmem SDPA path (#26331) 8b4b355 ci : move more jobs to ccache-buckets (#28375) 1863ac0 ui: export conversations from database instead of cached store (#27432) 49c0dc8 model : add Tencent Hy 4 (hy_v4) preview architecture support (#28127)

Source: llama.cpp Releases | 2026-09-14