b10785
metal : add sparse FA (#28098) metal : support n_kv_max sparse mask hint in flash attention vec kernel add kernel_flash_attn_ext_vec_idx: compacts finite mask entries into a per-row index list (Hillis
Knowledge catalogue
metal : add sparse FA (#28098) metal : support n_kv_max sparse mask hint in flash attention vec kernel add kernel_flash_attn_ext_vec_idx: compacts finite mask entries into a per-row index list (Hillis
mtmd: propagate const to preproc class (#28310) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44990064 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
tune MMVQ to MMQ crossover for SM87 (#28285) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45087167 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sil
opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations (speculative decoding/MTP) (#26477) opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations opencl: guard q4_K/q
common/json-schema : fix GBNF grammar generation for empty object schemas (#28279) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45102223 macOS/iOS: macOS
llama: fix whole source code rebuilt on each new commit (#28278) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45114012 macOS/iOS: macOS Apple Silicon (ar
metal : add metallib build support for xcframework (#28163) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44563342 macOS/iOS: macOS Apple Silicon (arm64)
hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (#28217) hexagon: fix bug ne[2] printed in proc_op_req prep-src log hexagon: add shape/VTCM farf logs to cpy, get/set rows, gdn Website
opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632) opencl: clamp the q4_K decode GEMV's fetch row on a padded x-grid opencl: enforce the tiling contract of the image KQ/KQV GEMMs open
vulkan : only request VK_KHR_shader_bfloat16 extension if supported (#28155) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44637403 macOS/iOS: macOS Apple
vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (#27449) vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec when NUM_COLS > 4. 5x perf at n=8 Assisted-by: Claude
hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (#28202) hex-mm: fuse QKV and FFN matmuls that land on HMX hex-mm: remove hardcoded ne[1] < 32K restriction hex-get-rows: explicitly reject repacked Q8
ggml: avoid KleidiAI buffer type init on dispatch (#27891) Co-authored-by: Acmmi acmmi@Acmmis-MacBook-Air.local Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestati
mtmd: Fix Qwen3-tts-0.6b (#28231) mtmd: load the qwen3-tts code predictor proj_in as optional The talker and the code predictor share the hidden size on the 0.6B checkpoints, so the reference builds n
mtmd: support DeepSeek-V4-Flash-Vision-Exp (#28133) mtmd: support DeepSeek-V4-Flash-Vision-Exp handle min/max token counts from CLI rm debugging use GGML_ROPE_TYPE_VISION nits apply review comments co
common, server : enable preserve_reasoning kwarg by default, log its effective state (#28174) common, server : enable preserve_reasoning kwarg by default, log its effective state If the preserve_reaso
ggml-cuda : remove unused vars (#28235) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44811308 macOS/iO
model: correctly support input vision for deepseek4 (#28154) model: correctly support input vision for deepseek4 nits Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/att
Update ROCm to 10.0.0 release (#27803) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44848715 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (
metal : fix memory query under low-memory conditions (#27701) metal: Fix memory query under low-memory conditions Simply variable name Co-authored-by: Georgi Gerganov ggerganov@gmail.com Write it even
qwen4exp: sum the indexer heads by slices (#28023) qwen4exp: sum the indexer heads by slices The head reduction went through a transpose and a sum_rows over ne[1], which left sum_rows with ne0 = 4, on
qwen4exp: support recurrent state rollback (#28123) MTP speculative decoding needs the target state to move back by the number of rejected draft tokens. Without rollback support the context is classif
ci: Bump ggml-org/ccache-action to v1.2.24 (#28083) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44375174 macOS/iOS: macOS Apple Silicon (arm64) macOS Ap
metal: enable Metal 4.0 tensor API on M5+/A19+ (#27461) metal : request Metal 4.0 language version for the tensor API metal : load the tensor API kernels from a separate metallib tests : add external-
tests : fix log verbosity for test-llama-archs (#28147) tests : fix log verbosity for test-llama-archs cont : naming cont : add note Website: https://llama.app Attestations: https://github.com/ggml-or
qwen4exp: fix seq_cp, block position keying, mtmd input, cuda abort, add tests (#27941) qwen4exp: follow up fixes -kvu NaN collapse fix Assisted-by: Claude indexer cache ext.x/ext.y restore fix Assist
sycl : support limit max alloc memory within 2GB for host-pinned memory (#27559) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44438607 macOS/iOS: macOS A
metal : add fa-vec tuning for M2 Max (#28015) Rows for M2 Max (30 GPU cores) collected with 'ggml-metal-tuning fa-vec --dtype f16,q8_0', pasted into fa_vec_tuned_table. ref: https://github.com/ggml-or
metal : fix more leaks due to missing autoreleasepools (#27883) metal : fix more leaks due to missing autoreleasepools metal : rename variable metal : fix another missing pool warning Co-authored-by:
model : load hparams.n_layer_nextn before n_layer() calls (#28159) load hparams.n_layer_nextn before n_layer() calls remove duplicate loads Website: https://llama.app Attestations: https://github.com/
metal : add fa-vec tunings for A18 Pro (MacBook Neo) (#28152) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44474754 macOS/iOS: macOS Apple Silicon (arm64
metal : add fa-vec tuning for M2 Pro (#28122) metal: add fa-vec tuning for M2 Pro metal : update fa-vec tuning for M2 Pro with new dtypes Website: https://llama.app Attestations: https://github.com/gg
context : autoscale n_ctx_train when yarn scaling specified (#28030) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44499127 macOS/iOS: macOS Apple Silicon
kv-cells: look up the n-gram history in the sequence position index (#28040) get_prev_tokens() rebuilt a (seq, pos) -> token hash map on every ubatch by walking all used cells, while llama_kv_cells al
cuda: fuse MoE weighted expert reduction (#25952) cuda : fuse MoE weighted reduction (mul + view + add) The MoE combine tail currently writes weighted expert outputs to global memory before reducing t
ggml: add SWIGLU_CLAMP (#27930) ggml: add SWIGLU_CLAMP add vulkan shader Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44034152 macOS/iOS: macOS Apple Sil
kv-cells: stop the sequence scan once all sequences are seen (#28011) for_each_token_in tested all LLAMA_MAX_SEQ sequences for every used cell, while a cell almost always belongs to one. The scan now
ggml : fix ggml_backend_buft_get_alloc_size() guard (#28038) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44038725 macOS/iOS: macOS Apple Silicon (arm64)
rpc: avoid serializing buffers from other servers (#26500) rpc: avoid serializing buffers from other servers Only include remote buffer pointers when the buffer belongs to the RPC dispatcher receiving
metal : add remaining Q4_1/Q5_0/Q5_1 fa-vec tunings for M2 (#28017) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44044108 macOS/iOS: macOS Apple Silicon
hexagon: fix CPY fence bug (#28033) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44046740 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm
vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (#28032) vulkan: add top-k radix sort shader for k >= 1024 add Qwen 3.8 Flash Next top-k tests add top-k qsa fusion clean up code Websi
ggml : add MUL_MAT to the list of ops that may need additional memory (for WebGPU) (#28071) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://githu
vulkan: tune mat-vec rows for batched inference on Strix Halo (#27909) vulkan: RDNA3 static mat-vec rows above four columns On RDNA3 above four columns a static 4 rows for all types benches faster tha
spec : fuse the DFlash encoder into the KV cache injection (#27310) dflash : fuse the encoder into the KV injection decode The encoder is a single fc + norm, but running it as a separate llama_encode
sycl : Enhance to get the free memory of Intel GPU (#27968) enhance get mem info by l0 an SYCL API remove debug code, format the code update SYCL.md for GGML_SYCL_GET_MEM_API Website: https://llama.ap
CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (#27621) CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion
metal : add fa-vec tunings for M1 (#28078) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44172949 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
ROCm: add radix TOP_K for long rows (#27466) ROCm: add radix TOP_K for long rows Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44178618 macOS/iOS: macOS A
webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_get() implementation (#28045) webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_ge
opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance (#26438) opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse opencl: Q4_K mul_mm 8x8 tile fot In
kv-cache : optimize restoring non-contiguous cells (#27991) kv cache : batch state restore scatter reads per contiguous run When restoring state into non-contiguous destination cells (e.g. a prompt-ca
AVX2: Speed up large batch size prompt processing of IQ models (#27402) Batched gemm for grid IQ quants Style updates and a bit more performance Clean up comments Move code around Vectorize IQ panel d
metal : add concat support for quantized types (#28116) Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/442
CUDA: XOR swizzle flash attn K,V smem fp16 tiles (#25635) CUDA: XOR swizzle flash attn K,V smem fp16 tiles Signed-off-by: ynankani ynankani@nvidia.com Fix use 64bit generic pointer instead of 32bit sh
metal : add fa-vec tunings for M1 Ultra (#28088) metal : add fa-vec tunings for M1 Ultra metal : move M1 Ultra tunings after M1 Max section metal : remove duplicate blank line Website: https://llama.a
metal : add fa-vec tunings for M2 (#27940) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43902029 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
ggml: allow passing alloc dependencies in graph_optimize (#27301) ggml: allow passing alloc dependencies in graph_optimize add alloc dep tests add TODO about using flat array Website: https://llama.ap
memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.c
metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (#25648) metal : fail closed on mul_mat shapes with missing F16 kernels metal : abort on nil pipeline in encoder_set_pipeline metal : ad