b11168
hexagon: dynamic quantizer improvements (#29395) hexagon: fix accuracy issue in Q8_0 N=1 MUL_MAT hex-quant: fix register spills hex-mm: use dma for all dyn.quant paths Co-authored-by: Aparna M P aparm
Knowledge catalogue
hexagon: dynamic quantizer improvements (#29395) hexagon: fix accuracy issue in Q8_0 N=1 MUL_MAT hex-quant: fix register spills hex-mm: use dma for all dyn.quant paths Co-authored-by: Aparna M P aparm
llama-grammar: fix numeric truncation for token_id parsing (#29382) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50007794 macOS/iOS: macOS Apple Silicon
hexagon: handle multi-sequence in concat_2d (#29344) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50014220 macOS/iOS: macOS Apple Silicon (arm64) macOS A
sync : ggml (#29396) ggml : bump version to 0.25.2 (ggml/1642) ggml : fix ubsan error in ggml_graph_nbytes (ggml/1644) ggml : bump version to 0.25.3 (ggml/1645) sync : ggml Website: https://llama.app
metal : optimize sparse FA + clean-up (#29377) metal : cache sparse FA indices in shared memory Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp metal : simplify shared memory size calculation A
HIP : optimize IQ2/IQ3 (__vsub4 __vcmpne4) using SWAR (#27962) HIP : use bit manipulation for __vcmpne4 HIP : use bit manipulation for __vsub4 Website: https://llama.app Attestations: https://github.c
hex-dma: introduce direct-mapped DMA cache that is better suited for HVX FA mask handling (#29282) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49404271
sampler: reduce the size of the probe (#29285) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49443624 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
vulkan: hide internal symbols to prevent duplicate-dlopen state destruction (#29139) Since #28732 our internal symbols are exported. A duplicate copy dlopened and dlclosed by ggml_backend_load_all() t
sycl : support op get_rows_back, only support fp32/fp16 (#25266) resovle confict support gedt_rows_back, update the ops.md Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cp
sycl: extend MMVQ GLU fusion, add rms_norm+scale and ssm_conv+silu fusions (#28931) sycl : extend MMVQ GLU fusion to mixed quant types; add rms_norm+scale and ssm_conv+silu fusions fixing spacing issu
sycl : support new UT case for mul_mat_hadamard fp16 (#29218) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49484593 macOS/iOS: macOS Apple Silicon (arm64
cuda: top-k MoE should always fire (#28432) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49495074 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sili
ggml-meta: resolve multi buffer views (#29266) ggml-meta: resolve multi buffer views add TODO to revisit if graph allocator gets refactored Website: https://llama.app Attestations: https://github.com/
vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415) vulkan: optimize IQ4_XS matmul kernels Assisted-by: OpenAI Codex vulkan: address IQ4_XS review nits drop the dead LOAD_VEC_A != 8 branch in the IQ4_X
make-release : update summary prompt Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49525233 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (ar
model : support Gemma4 DSpark draft backbone (#29226) dspark: add Gemma 4 draft support Add GGUF conversion and runtime support for full-attention and SWA Gemma 4 DSpark drafts, including tied output
server: Dedup the draft HF model via dedup-cache-models (#27934) server: Dedup the draft HF model via dedup-cache-models Fixes #27846 server: avoid capturing structured binding in lambda Website: http
server: accept OpenAI video_url content type and data: video URIs (#27921) The OpenAI chat completions API specifies content part type 'video_url' with a {'url': ...} object, and clients typically sen
jinja : parse unary +/- before variables (#29244) jinja : parse unary +/- before variables Lexer already emits unary_operator for -n / +n, and runtime executes unary -. Parse them at multiplicative pr
server: fix token counting API crash on sleep (#29309) server: wake up sleeping server correctly server: wake up sleeping server correctly (local aliases removed) Website: https://llama.app Attestatio
CUDA: enable sparse-fa for dsv4 prefill (again) (#29298) CUDA: enable sparse-fa for dsv4 prefill (again) CUDA: unroll the query loop of the sparse mask scan The query loop of flash_attn_mask_to_sparse
llama.cpp : bump version to 0.5.0 (#29333) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49623059 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
opencl: add A8 Q6_K non-MoE dp4a binary kernel (#29057) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49630180 macOS/iOS: macOS Apple Silicon (arm64) macO
tests: add -b/--backend option to test-llama-archs for testing a specific backend (#27372) tests: add backend option to test-llama-archs Update tests/test-llama-archs.cpp Co-authored-by: Johannes Gäßl
Overview This release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation. It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR
metal : fix mask bounds in flash attention block pre-pass (#29220) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49087452 macOS/iOS: macOS Apple Silicon (
vendor : update cpp-httplib to 0.57.1 (#29239) Signed-off-by: Adrien Gallouët adrien@gallouet.fr Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49092442 ma
hexagon: new HMX-optimized GATED_DELTA_NET (#29199) hex-gdn: start putting together HMX support for GDN hex-gdn: working hmx but not-pipelined and slow for now hex-gdn: re-write vtcm layout handling a
opencl: add A8 Q4_0 non-MoE dp4a binary kernel (#29055) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49141307 macOS/iOS: macOS Apple Silicon (arm64) macO
ci : publish snapdragon builds in release workflow (#29007) The snapdragon CI builds packages only to feed the QDC device tests, so Hexagon NPU binaries never reached the releases page. Build both tar
chat : Fix Muse Glimmer tool-call first parser error (#29242) Fix Muse Glimmer tool-call first parser error Add test to verify Condense patterns remove test for trigger patterns Website: https://llama
cmake : allow repeated find_package calls for llama (#29228) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49213556 macOS/iOS: macOS Apple Silicon (arm64)
convert: add MiMo-V2.6 support (#29257) convert: add MiMo-V2.6 support Hoist the K3 mxfp4 conversion repack into base.py so it can be reused Remove decoder from mmproj convert Update conversion/mimo.p
spec : support DFlash for HunyuanOCR (#28890) model : add DFlash layer-input taps for HunyuanVL DFlash speculative decoding needs the target graph to expose the residual stream entering each layer (re
server: Add support for binding to multiple addresses (#28690) Add support for binding llama-server to multiple addresses Assisted-by: Codex remove redundant thread handler make it clear about overlap
jinja: use const for statement::execute and ::visit (#29271) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49279983 macOS/iOS: macOS Apple Silicon (arm64)
ggml : IQ1_M build prefix sums once per block (#28706) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49294443 macOS/iOS: macOS Apple Silicon (arm64) macOS
metal : gate mul_mm_id src1 rescale behind ggml_prec (#29029) metal : gate mul_mm_id src1 rescale behind ggml_prec Assisted-by: Claude Fable 5.1 ggml-webgpu: reject MUL_MAT_ID when src1 precision is F
mtmd: add various sanity checks (#29276) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49308522 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon
vulkan: add Intel Xe flash attention optimization kernels (2/3, Xe-LPG Plus/Xe2/Xe3) (#24406) vulkan : Intel FA kernel optimization for split k path vulkan : Host code update for Intel split k FA kern
server: support input_image in function_call_output (#20663) (#22575) server: support input_image in function_call_output (#20663) server: fix if statement spacing server: avoid repeated type lookup W
server: do not pass log file to children (#29212) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49344105 macOS/iOS: macOS Apple Silicon (arm64) macOS Appl
server: fix router eviction races with the existing queue (#29217) server: route every model load through the queue A model loaded by the fast path has no queue entry, so tick() evicts it at its LOADE
opencl: add bin kernel kernel_gemm_noshuffle_q4_k_q8_1_dp4a_ila_a8_bin (#29056) opencl: add A8 Q4_K non-MoE dp4a binary kernel opencl: rename binary kernel selection helpers Website: https://llama.app
webgpu : add fused gdn + cpy (#28976) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48873164 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (a
metal : fix deprecation warnings from macOS 27 SDK (#29136) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48890258 macOS/iOS: macOS Apple Silicon (arm64)
cuda : tune MMVQ to MMQ crossover for SM70 (Volta) (#28912) tune MMVQ to MMQ crossover for SM70 (Volta) Signed-off-by: Yangyu Chen cyy@cyyself.name Apply suggestion from @JohannesGaessler Apply sugges
hexagon: overhaul of buffer and DMA handling to support 64bit mappings + improvements (#29197) hex-dma64: enable support extended buffer mappings and 64bit dma hex-dma64: expand binary ops to support
ci : Upgrade CUDA to 13.4 for Ubuntu CUDA Release Builds (#29202) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48951751 macOS/iOS: macOS Apple Silicon (a
sycl : coalesce MKL-FA softmax loads instead of one work-item per row (#28918) sycl : coalesce MKL-FA softmax loads instead of one work-item per row better human readable variable name Website: https:
json: Fixed json enum handling (#28518) Fixed json enum handling Added common_json_value handling for enum values. Added tests/test-json.cpp to cover testing of some aspects of common_json. Removed te
ggml-metal : simplify fusion pattern op list declaration (#29206) ggml-metal : derive non-empty fusion ops from ops_all Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp ggml-metal : drop _all su
tests : remove stale comment (#29140) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48998612 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (a
server : do not forward --api-key-file to router-spawned child instances (#28938) In router mode, authentication belongs to the router. unset_reserved_args() already unset LLAMA_API_KEY, but did not u
args: add env vars for temperature, top-p, min-p and penalties (#27380) Allow configuring --temp, --top-p, --min-p, --repeat-penalty, --presence-penalty and --frequency-penalty via LLAMA_ARG_* so llam
tests/test-backend-ops : allow regex entries in the -o filter (#29204) tests/test-backend-ops : allow regex entries in the -o filter so far -o only accepted a comma separated list of exact op names or
test-llama-archs : make tensor data stdev configurable and improve help (#29133) test-llama-archs : make tensor data stdev configurable Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp test-llam
cuda: fix sm_70 tile compilation error (#29224) The 5-argument load_ldmatrix added in 1884824 only defines tile<16,8>, so the Volta tile<8,4> does not match. See #29222 for details. Building on 188482
chat : add dedicated Ling 3.0 (Bailing V3) parser (#28682) chat: add dedicated Ling 3.0 (Bailing V3) parser Ling 3.0 Flash templates pre-open the think block in the generation prompt, so the model nev