b10681
vulkan: Change mul_mat_id to pad K rather than N (#27925) The N padding is needed for mul_mat, but not mul_mat_id. For mul_mat_id, we indirect the row index through a shared memory lookup table which
Knowledge catalogue
vulkan: Change mul_mat_id to pad K rather than N (#27925) The N padding is needed for mul_mat, but not mul_mat_id. For mul_mat_id, we indirect the row index through a shared memory lookup table which
metal : add fa-vec tunings for M1 Max (#27932) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43859747 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (#27526) vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs remove one more fastdiv W
sycl: make --fit respect --fit-target better (#27629) improve the --fit algorithm to take into account the actual peak required VRAM for a given context size on a SYCL backend. This includes both prop
metal : add remaining fa-vec tunings for M4 Pro (#27915) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43865923 macOS/iOS: macOS Apple Silicon (arm64) mac
metal : assert shared memory padding (#27951) metal : assert shared memory padding cont : add ref Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43867822 m
opencl: use a better matmul path on two Adreno GPU generations (#27640) opencl: default the Adreno xmem F16xF32 GEMM on for X2E kernel_mul_mm_f16_f32_l4_lm is the slowest matmul this backend has on Ad
model: add DSpark support for Nemotron3.5 (#27804) model: add DSpark support for Nemotron3.5 Update src/models/dflash.cpp Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-
tests : run test-save-load-state across all architectures (#27755) tests : run test-save-load-state across all architectures test-save-load-state previously only ran in ctest against a single download
metal : add fa-vec tunings for M4 Pro (#27824) This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discussions/27668 to add support for additional Apple GP
metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (#27863) metal : add fa-vec tunings for M5 This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discuss
sycl: bind the f16 KV cache in place for the oneDNN SDPA path (#27468) Measured at a live KV length of 34816 (32768 depth plus one 2048 ubatch), on Qwen3.8 27B Q4_K_S: per tensor 4 * 34816 * 256 * 2 B
sycl: use TILE for quantized KV decode on BMG (#26689) Route quantized KV decode to TILE on Xe2 (BMG) only, keep VEC on other archs until validated there. Website: https://llama.app Attestations: http
OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (#27843) OpenVINO Backend: Fuse IM2COL + MatMul convolution into OpenVINO convolution ci:ggml-ov: Skip recurrent state
metal : add fa-vec tunings for M4 (#27875) This adds fa_vec_tuned_table records for Apple M4 to ggml-metal-tuning.cpp. Includes F16, Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0. (M4, 10 GPU Cores) Co-authored-by
Vulkan: add hoisting support for row IDs and expert count in shaders (#26686) vulkan: add hoisting support for row IDs and expert count in shaders use hoisted row ids in coopmat2 vulkan: address revie
ggml : fix conv_transpose_2d for multiple batches (#26132) ggml : fix conv_transpose_2d for multiple batches ggml_compute_forward_conv_transpose_2d_impl only computed the first batch (ne[3] of the des
vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812) vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize is_src_of doesn't treat two views of one tensor as
model: qwen4exp: reduce number of graph splits (#27880) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43734155 macOS/iOS: macOS Apple Silicon (arm64) macO
bench: add --tensor-read-lazy (#27881) bench: add --tensor-read-lazy rm the alias rename to LLAMA_LAZY_MODE_* Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestation
hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (#26501) hexagon: use non-host bufs by default and make the backend fully async hex-hb: remove optional hostbuf suppor
models : support nanbeige4.2-3B (#27730) Co-authored-by: admin lizongqiang@kanzhun.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43308396 macOS/iOS: m
llama : add --n-cpu-ffn option (#26622) common : dedupe --n-cpu-moe / --spec-draft-n-cpu-moe override loops common : add --n-cpu-ffn to CPU-offload dense FFN weights of first N layers common : general
metal : fix memory leaks due to missing autoreleasepools (#27758) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43372015 macOS/iOS: macOS Apple Silicon (a
args: add --video-* CLI arguments (#24318) args: add --video-* CLI arguments gen docs nits add mtmd_helper_init_opt Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attes
model : simplify MiniMax-01 graph (#27790) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43387702 ma
spec: Add benchmark-only synthetic speculative acceptance options (#27711) Add benchmark-only synthetic speculative acceptance to llama-server and llama-cli Address review comments Address review comm
llama: model_loader: add TENSOR_READ_LAZY (#27794) llama: model_loader: add TENSOR_GET_ROW_LAZY add --tensor-read-lazy rename to TENSOR_READ_LAZY gen docs address comments Website: https://llama.app A
Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan Backend (#27453) vulkan: add LIGHTNING_INDEXER op vulkan: updated lightning_indexer.comp and ggml-vulkan.cpp with 128-lane dot-pr
quantize: cap working memory size to avoid loading big tensors onto RAM (#27795) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43468853 macOS/iOS: macOS A
opencl: add bin kernels kernel_gemm_moe_q4_0_q8_1_dp4a_bin, kernel_gemm_moe_mxfp4_q8_1_dp4a_bin (#27768) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/434
spec : add DFlash2 support (local convolution + candidate selector) (#27342) (#27816) spec : add DFlash2 support (local convolution + candidate selector) (#27342) support DFlash2 Add p_min in DFlash2
ci : bundle HIP runtime DLLs with Windows ROCm release (#26973) Copy amdhip64_7, amd_comgr and rocm_kpack next to the binaries so the correct HIP runtime loads over the driver's copy in System32. Fixe
model: add Qwen3.8-Flash-Next (qwen4exp) (#27742) gguf: add qwen4exp (Qwen3.8-Flash-Next) arch and converter Adds the GGUF-side plumbing for HF model_type qwen4_exp: MODEL_ARCH.QWEN4EXP plus tensors f
ci : build only the ggml-hip backend for windows-rocm release (#27753) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43506547 macOS/iOS: macOS Apple Silic
server: add ctx-per-slot (--kv-unified-per-slot) (#24124) Add ctx-per-slot argument for unifid KV cache Swap out ctx fractions for ctx pool slots Formatting cleanup Remove ctx-pool-slots, make ctx-per
hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (#27798) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43526462 macOS/iOS: macO
ggml-hexagon: add HTP unary ops for ABS and LOG (#27786) Add HVX-accelerated implementations for GGML_OP_LOG and GGML_UNARY_OP_ABS on the HTP backend. Register HTP_OP_UNARY_ABS and HTP_OP_UNARY_LOG in
ggml-meta: propagate buffer usage and call init on the new tensors (#27586) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43044002 macOS/iOS: macOS Apple
ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization (#26647) metal: WIP chunked SSD SSM_SCAN kernels for multi-token prefill metal: drop scalar SSD path; MMA + sequential tail drop WIP ss
cuda: unblock mmq for MoE on sm_60 (#26264) cuda: unblock mmq for MoE on sm_60 cuda: duplicate mmq-config-pascal for dp4a and older cuda: reduce occupancy on non-dp4a pascal for Q2_K, Q4_K, Q5_K, Q6_K
ci: Clean up UI builds from releases (#27706) ci : inline UI version resolution into ui-build.yml ci : build UI once and reuse the artifact in release jobs Server jobs now extract the ui-build artifac
Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (#27216) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43166320 macOS/iOS: macOS Apple S
vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps (#27726) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43173045 mac
llama: add token ID tracking to KV cell (#27762) kv: track token id rm get_prev_tokens, move it to the main pr nits add get_prev_tokens Website: https://llama.app Attestations: https://github.com/ggml
sycl : mark tq2_0 as not supported (#27660) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42777546 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sili
grammar : parse - in char classes as literal hyphen (#27591) grammar : accept '-' escape in character classes gbnf_escape_char_class() escapes '-' as '-' but parse_char() rejected that escape, so gene
sync : ggml Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42810281 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DI
llama.cpp : bump version to 0.3.0 (#27696) llama.cpp : bump version to 0.3.0 ci : update release default desc scripts : add prompt for generating release summary Website: https://llama.app Attestation
metal : null-check buffer alloc to fix OOM crash (#25371) metal : null-check ggml_metal_buffer_init result to avoid OOM crash ggml_backend_metal_buffer_type_alloc_buffer used the result of ggml_metal_
chat : scope qwen3-coder workarounds (#27679) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42919681 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Si
rpc: support apple RDMA as an RPC transport (#26421) rpc: support apple RDMA as an RPC transport remove set_tensor micro optimization, rpc socket pinning per CR remove transparent reconnect trigger ap
ci : update OS used for ROCM to Ubuntu 24.04 (#27681) This matches what other build targets use and also what AMD advertises wheels as supporting. Website: https://llama.app Attestations: https://gith
kleidiai: Rework KleidiAI Build System/Integration (#26077) Rework KleidiAI Build System/Integration Signed-off-by: Jonathan Clohessy Jonathan.Clohessy@arm.com Add fp16 guard, and fix cmake caching is
Overview llama.cpp 0.3.0 introduces the dots3-note multimodal model (with a new DSA-ISWA KV cache), MTP support for GLM-4.5-Air, and tensor-split (-sm tensor) plus multi-sequence rollback fixes for De
Deepseek 4: -sm tensor (#26490) DSV4: sm tensor set coarser granularity for head splits fix dspark add model saving for dsv4 + allow dflash to return on specific device add comment about dsv4 seq_rm s
mamba2 : Flatten in/out projections to dispatch GEMM instead of GEMV (#27513) mamba2 : flatten mamba2 in/out projections to dispatch gemm instead of gemv mamba2 : remove redundant output reshape Websi
ggml : fix ggml_clamp (#27644) ggml : fix ggml_clamp cont : update ggml-alloc Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42576162 macOS/iOS: macOS Appl
mtmd: video: fix moov atom at the end of file (#27596) mtmd: video: fix moov at the end of file Co-authored-by: rkfg rkfg@rkfg.me fix SIGPIPE windows: handle broken pipe case Co-authored-by: rkfg rkfg
ggml : shorten virtual device naming in CUDA and Metal (#27608) ggml : shorten virtual device naming in CUDA and Metal Assisted-by: llama.cpp:DeepSeek-V4-Flash-0731 ggml-metal : build device descripti