Model Releases
v0.5.0
Overview This release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation. It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR
Overview This release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation. It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR conversion support, ggml 0.25.0 backend improvements, multi-address HTTP binding, image outputs from function calls, and several chat parser/UI fixes. Highlights Accelerate CUDA conv2d with implicit GEMM (#29135) Add Metal MoE and SSM_CONV fusion optimizations (#28948) Allow the server to bind to multiple addresses (#28690) API changes Add llama_adapter_lora_init_from_file_ptr() for loading LoRA from an open FILE (#28993) Document llama_model_load_from_file_ptr() as reading from the current position and requiring aligned mmap (#28993) Add LLAMA_VOCAB_TYPE_TEST dummy tokenizer (#29084) Add input_image support to server function-call outputs (#22575) Allow --host to accept comma-separated TCP addresses and UNIX sockets (#28690) New models Add HRM-Text / DFM Mimir 1B support (#27625) Add MiMo-V2.6 conversion support (#29257) Add DFlash support for HunyuanOCR (#28890) Extend Nemotron MTP and Nemotron-H model handling (#29018, #28989) Add Qwen4Exp hyper-connection ops and sparse flash attention (#28901, #28770) Add --fuse-qkv support for Muse Glimmer (#29203) Core changes Add graph input/input-tensor diagnostics during scheduler reserve (#26625) Enable CUDA graphs for MTP drafting (#28549) Fix tensor-parallel split state/granularity for fused QKV models (#28965) Fix Mamba time-step projection input contiguity (#28832) Write the SWA pattern in the model saver and round-trip 15 more architectures (#29042) Add environment variables for temperature, top-p, min-p and penalties (#27380) Reduce the sampler backend probe size (#29285) Add Ling 3.0, DeepSeek V3.2/V4, qwen3-coder, Muse Glimmer and Gemma 4 parser fixes (#28682, #29008, #28869, #29242, #29115) Improve JSON Schema and PEG handling (#28518, #29127, #29161) Add ufakzeka pre-tokenizer and llama-bench --version (#29033, #28971) Multi-modality changes Add sanity checks for mtmd layer indices, SAM layer counts, resize targets and graph allocation (#29276, #28149) Fix SigLIP bucket buffer overrun for tall/wide images (#29276) Server changes Fix router eviction races and child process lifecycle handling (#29217) Do not pass log file or API key file to router-spawned children (#29212, #28938) Improve startup and model-source logging (#29125) Update vendored cpp-httplib to 0.57.1 (#29239) UI changes Accept WEBM video files (#28622) Add close button to UI toasts (#28246) Fix mobile breakpoint and content overflow issues, including horizontal table scrolling (#29108) Restore the reasoning menu in single-model desktop mode (#27985) Stop re-probing a disabled /tools endpoint on every message (#28646) ggml changes Updated ggml to v0.25.0 (release) The release expands hyper-connection, flash-attention, and fused MoE/SSM support across backends, with robustness, quantization, data-layout, and RPC/meta improvements. API changes include gated ggml_dsv4_hc_pre_gated(), optional ggml_dsv4_hc_post() comb, and RPC protocol major v7. Assets Nightly build: b11146 More info Releases and versioning of ggml-org projects Changelog since v0.4.1 7fe450e llama.cpp : bump version to 0.5.0 (#29333) 177cd8c sync : ggml e4e2f62 ggml : bump version to 0.25.1 (ggml/1637) 66fba63 CUDA: add a reserve to avoid spurious warning on older GCC builds (#29317) bddf826 common : keep HF cache dir as path, expose UTF-8 only for logs (#29320) 9575389 metal: add the missing f32 x bf16 mul_mv variants (#28741) dc9879c CUDA: enable sparse-fa for dsv4 prefill (again) (#29298) 42916d8 server: fix token counting API crash on sleep (#29309) 4e416ee jinja : parse unary +/- before variables (#29244) ee3ecce metal : key the fa-vec tuned table by family instead of SKU (#29075) 057494f server: accept OpenAI video_url content type and data: video URIs (#27921) bcbc936 server: Dedup the draft HF model via dedup-cache-models (#27934) 26758d3 ci : fix build-cmake runner target (#29299) 18f9f7b model-conversion : add causal-compare-logits recipe (#29305) 633733d model : support Gemma4 DSpark draft backbone (#29226) 86b2daa ci : run python (jinja) test (#29302) 183d2a0 make-release : update summary prompt 45062d4 sync : ggml 503549c ggml : bump version to 0.25.0 (ggml/1635) e97545d sycl : fix compile warnings b1ff4ca vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415) 9425611 ggml-meta: resolve multi buffer views (#29266) 1a67982 cuda: top-k MoE should always fire (#28432) 384a534 sycl : support new UT case for mul_mat_hadamard fp16 (#29218) 5e48b31 sycl: extend MMVQ GLU fusion, add rms_norm+scale and ssm_conv+silu fusions (#28931) 4d7d770 sycl : support op get_rows_back, only support fp32/fp16 (#25266) 08b1d2a vulkan: hide internal symbols to prevent duplicate-dlopen state destruction (#29139) 441df11 sampler: reduce the size of the probe (#29285) e6ab7c1 hex-dma: introduce direct-mapped DMA cache that is better suited for HVX FA mask handling (#29282) f46bc30 HIP : optimize IQ2/IQ3 (__vsub4 __vcmpne4) using SWAR (#27962) 709fe75 jinja : fix dangling reference warning in for_statement (#29279) d5f6649 opencl: add bin kernel kernel_gemm_noshuffle_q4_k_q8_1_dp4a_ila_a8_bin (#29056) 9919911 server: fix router eviction races with the existing queue (#29217) bbf99b1 server: do not pass log file to children (#29212) 4098fdc server: support input_image in function_call_output (#20663) (#22575) 4ceb171 vulkan: add Intel Xe flash attention optimization kernels (2/3, Xe-LPG Plus/Xe2/Xe3) (#24406) 73c941b mtmd: add various sanity checks (#29276) 0f8a414 metal : gate mul_mm_id src1 rescale behind ggml_prec (#29029) f95b0d9 ggml : IQ1_M build prefix sums once per block (#28706) c350a40 Performance tune for gemma4-26b-a4b flash attention shape. (#28450) 9b421fa ui : Accept WEBM video files (#28622) 348f853 jinja: use const for statement::execute and ::visit (#29271) 217f81c server: Add support for binding to multiple addresses (#28690) 828fdf2 spec : support DFlash for HunyuanOCR (#28890) bfd73a8 convert: add MiMo-V2.6 support (#29257) a60f9ae cmake : allow repeated find_package calls for llama (#29228) 7ab4ee7 chat : Fix Muse Glimmer tool-call first parser error (#29242) 0ee9435 ci : publish snapdragon builds in release workflow (#29007) 8cfc315 Add close button to UI toasts (#28246) ec5a12b opencl: add A8 Q4_0 non-MoE dp4a binary kernel (#29055) c550d2f ci : update Level Zero SDK to v1.33.1 and enable the L0/oneDNN CMake flags in the SYCL job (#29230) 5836771 hexagon: new HMX-optimized GATED_DELTA_NET (#29199) ff0dbb9 vendor : update cpp-httplib to 0.57.1 (#29239) fb34fc2 metal : fix mask bounds in flash attention block pre-pass (#29220) c641dfa test-save-load-state : compare logits with NMSE and feed expected tokens (#29238) 9655061 llama-context : report graph inputs and input tensors during sched reserve (#26625) b1c2863 cuda: fix sm_70 tile compilation error (#29224) f4e276a ggml-cuda : convert contiguous tensors four elements at a time (#29155) e6cef81 cuda : accelerate conv2d with implicit GEMM (#29135) c21284c ggml : fix dimension and stride truncation in ggml_permute (#29227) 6f41ac5 vendor : update cpp-httplib to 0.57.0 (#29214) ec91ab5 docker : bump cuda to 13.4.1 (#29207) bb3c853 sycl : support gated DSV4_HC_PRE and optional HC_POST comb matrix (#29132) af91114 sycl : pinned memory use right device context instead of 0 (#28895) 1884824 CUDA: Follow up of #25635, refactoring FA shared smem swizzle (#28536) 161755f test-llama-archs : make tensor data stdev configurable and improve help (#29133) 1d72b05 tests/test-backend-ops : allow regex entries in the -o filter (#29204) 542e920 ci : refactor build-self-hosted into backend-specific workflows (#28991) e0dff58 args: add env vars for temperature, top-p, min-p and penalties (#27380) 982a332 server : do not forward --api-key-file to router-spawned child instances (#28938) 711f60b tests : remove stale comment (#29140) 335b21f ggml-metal : simplify fusion pattern op list declaration (#29206) 26394b4 json: Fixed json enum handling (#28518) 1aa2954 sycl : coalesce MKL-FA softmax loads instead of one work-item per row (#28918) 8034c1d ggml-cpu: ARM Repack kernels for Q1_0 (#23492) 6ad1af5 ci : Upgrade CUDA to 13.4 for Ubuntu CUDA Release Builds (#29202) 0c3626e hexagon: overhaul of buffer and DMA handling to support 64bit mappings + improvements (#29197) 68d9053 cuda : tune MMVQ to MMQ crossover for SM70 (Volta) (#28912) 8aa161b metal : fix deprecation warnings from macOS 27 SDK (#29136) 932a68e webgpu : add fused gdn + cpy (#28976) 62668d6 convert: enable --fuse-qkv for muse-glimmer (#29203) ce8caa6 CUDA: tune FA for Gemma 4 on Ampere or newer (#29152) a894dae metal : support arbitrary hc in dsv4_hc_pre (#29169) 3d82ef6 common/peg : handle invalid utf-8 sequences in the AST (#29161) 3cf0325 CUDA: enable sparse fa for qwen4 (#28770) b23efaa ui: Fix mobile breakpoint + content overflow issues (#29108) 4260903 fix(mamba) : make time-step projection input contiguous (#28832) 9a9f939 metal: add F16 input to the FWHT (#29094) f072b10 chat : fix gemma4 required tool grammar (#29115) 59657a6 chat : add dedicated Ling 3.0 (Bailing V3) parser (#28682) e613ef2 hexagon: enable I32 GET_ROWS (#29116) 851cb34 hexagon: add support for GEGLU_QUICK (#29114) 7d4b92b hexagon: enable support for TOP_K op (#29113) 1af554f server : improve startup log messages (#29125) eb1e1f4 json-schema : accept escaped hyphen in regex patterns (#29127) 5b59b83 metal : add MoE and SSM_CONV fusion optimizations (#28948) 60b06ab metal : fix FA support checks (#29122) efa28e9 test-llama-archs : generate dummy test vocab (#29084) 59fc5a1 metal : support qwen4exp hc ops (#29000) b23701f cuda : fix CUB argsort corruption caused by in-place keys (#28389) 60081bb opencl: add support for bin kernel flash_attn_f32_f16_bin (#29046) 2b18470 hexagon: add ROLL op support (#29105) 50631b3 hexagon: im2col update (#29103) 18a04f0 hexagon: HMX flash-attention head_dim padding (support DK=DV=72) (#26539) ec92815 opencl: add bin kernel kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin (#28678) 4fea119 ggml-cpu: add F16 input to the FWHT (#27779) 5b335f4 ggml : check for allocation failures to prevent crashes (#28149) 542348a Model-Saver: Write the SWA pattern, 15 more architectures roundtrip (#29042) d663dd3 ci: change ubuntu-latest to ubuntu-24.04 (#29079) 44be98f ggml-webgpu: fix supports_op condition for GET_ROWS (#28978) 911f6cd ggml : handle graph buffer reservation failure (#26070) bbd488c vulkan: add IQ3_S MMQ matmul kernels (#28822) dc85f89 vocab : add ufakzeka pre-tokenizer (#29033) 8ed1a55 cmake : fix build when GGML_CPU=OFF and GGML_CUDA=ON (#29026) bb11ebb gguf-py: fix Q8_1 block size in GGML_QUANT_SIZES (2+2+32) (#29036) f03cf3e ci : disable GHA cache for copilot (#29068) bdcbaaf ci : bump android-actions/setup-android to 4.0.4 (#29065) 5c53396 vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 512 experts (#28501) 972d231 ci : add missing evict-old-files (#29041) c77ae69 rpc : skip ACCEL devices (#29020) b49650a model : skip gate_up_exps if TENSOR_SKIP is set (#29014) 7076180 model : extend Nemotron MTP support (#29018) ebbb185 openvino : Update OpenVINO to 2026.4;fix clangd,MSVC warnings; (#29009) 4ff829e ui: fix removed reasoning menu in single model mode on desktop (#27985) f172be7 vulkan: split buffers and debug code into separate files, add shared headers (#28732) 87f9c82 ci : add API/ABI check to make-release workflow [no ci] (#28947) 7f6f0c2 chat : add message delimiters to the DeepSeek V3.2/V4 parser (#29008) 81aeaeb gguf : align the data section relative to the GGUF start, not the file (#28993) c9a5eee sycl : fix the B70 mem allocate error when >19.3GB (#28953) 7490357 vulkan: skip unneeded MoE work in mul_mm coopmat1 path (#25483) 817e5f8 sycl: ssm_conv: fuse the SiLU epilogue into the ssm_conv kernel (#28929) c57da6f opencl: fix various warnings (#28984) 79bfc1d docs: remove JG as CODEOWNER for test-llama-archs (#29003) 05f2dcf vulkan: fix buffer_reference alignment in im2col shaders (#28996) 35822af vulkan: support qwen4exp hc ops (#28988) aa39d7a [SYCL] Fix function signature for ggml_backend_sycl_split_buffer_type (#28981) 4bc272f vulkan: work around NV bug with argsort_large.comp (#28975) fb27a52 TP: fix split state and granularity for fused QKV gemma4, qwen35 (#28965) c6824a9 ci: switch fast jobs back to github (#28959) 2f3fd02 Enable CUDA graph for MTP draft (#28549) 1ec8188 hexagon: Support for K-Quants Q4_K and Q6_K (#28994) 82324fc hexagon: accept the zeroed rope probe in supports_op (#28995) 7ceed87 models : allow Nemotron-H models to only define layer_norm_epsilon (#28989) 7d6f5d0 model : add support for HrmTextForCausalLM (DFM Mimir 1B) (#27625) 83078fe CUDA/HIP: improve access patterns in im2col (#28013) f266648 spacemit : fix wrong transpose function for int16 data (#25161) 6019933 rpc : invalidate cached compute graph when a referenced buffer is freed (#24292) b04d4e5 Change max context length for auto-fitting with unified KV (#28849) 37b53fd qwen4exp: add hc ops (#28901) fccf716 HIP: broaden MoE ncols_opt tile heuristic on RDNA3.5 architecture (#28935) 0bec16e chat : force n on reasoning budget end for qwen3-coder (#28869) d4365d9 vulkan: make MUL_MAT_ID BN/2 tail unconditional (#28923) 0a8b29a metal: fix NaN in mul_mm_id when activations exceed f16 range (#26223) 583926e ci : add self-hosted webgpu to hf-jobs (#28712) e13469a llama-bench: support --version to print build info (#28971) 930e2fa hexagon: add back missing contiguous fast-path and hvx_copy_uu for each run (#28886) 72b590d hex-cpy: use dma if src and dst are contiguous (#28906) 38a5b42 HIP: Enable AllReduce for ROCm (#27825) 9f31776 opencl: choose the MoE expert matmul by batch size for speculative decoding/MTP (#27637) d1d3c33 ci: build MUSA for only 1 arch (#28944) 6011c34 docs: Rule of thumb for AI review time [no ci] (#28945) 7609846 rpc : hash-cache only weights (#28789) 5431581 cuda: support row-contiguous SUM_ROWS (#26308) 9e71716 models : move build_arch_graph() after graph() template specialization (#28934) fc82583 vulkan: support sparse Flash Attention (#28105) 77d554b OpenVINO: optimize stateful decode and GPU MoE inference (#28638) 6ec1a7e opencl: add generic ssm_scan (#28881) 1af6c65 ci: bump kleidiai runners from 22.04 to 24.04 (#28885) 1e7bcf3 metal : add FA kernels for HSK=96, HSV=64 (MiniCPM3) (#28599) 0ecb159 ci: Bump CUDA Windows x64 builds to 13.4.1 (#28930) 987498f ci : fix android release (#28936) 4c9233c cuda : enable i16 and i32 for DUP (#28897) 69eb250 cmake : use PROJECT_SOURCE_DIR instead of CMAKE_SOURCE_DIR (#28771) 1bc7a5a webui: stop re-probing disabled /tools endpoint on every message (#28646) 7cf1c54 ci : reuse build tag name when used instead of safe one (#28911) 96ffdc4 CI: hip-quality-check: ignore spill added in bfdc321 (#28909) bfdc321 HIP: fattn-mma: use fp32 accumulation on MFMA devices (#28576) 391fac1 ci : add ubuntu-cuda builds to release (#28186) 41abbfd qwen4exp: enable rms_norm + mul fusion (#28896) b4fa47d release : added gfx1103 to ubuntu rocm build (#28423) f3a184b cmake : remove precompiled headers (#28892) dfe4516 scripts: Add script to verify API/ABI compatibility (#28579)
Source: llama.cpp Releases | 2026-09-23