b11433
hexagon: add pool op support (#29995) hexagon: add pool_2d support hexagon: add pool_1d support hex-pool: dma changes hex-pool: Optimize HTP pooling boundaries and DMA pipelining hex-pool: code cleanu
Knowledge catalogue
hexagon: add pool op support (#29995) hexagon: add pool_2d support hexagon: add pool_1d support hex-pool: dma changes hex-pool: Optimize HTP pooling boundaries and DMA pipelining hex-pool: code cleanu
Overview llama.cpp v0.6.0 introduces the new llama_batch_ext extended batch API (with llama_process) for mixed token/embedding inputs and MTP/deepstack state embeddings, adds support for the GLM-5.3-F
webgpu: add f16 support to fill/set_rows (#29897) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52497832 macOS/iOS: macOS Apple Silicon (arm64) macOS Appl
ci : pushing tag needs deploy key (#29937) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52545484 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
common : prepare load_from_models_dir() for path conversion (#29674) This is part of the fs::path modernization series. That was also the opportunity to remove fs_list(). Signed-off-by: Adrien Gallouë
spec : fix n-gram drafts rejected at temp > 0 after truncation (#29924) Co-authored-by: Pranesh Gonegandla pgonegandla@nvidia.com Website: https://llama.app Attestations: https://github.com/ggml-org/l
imatrix: calculate activation-based statistics for new format (GGUF) imatrices (#14891) Use activations to calculate the stats Determine calculation mode Compute entropy for activations Compute cosine
vulkan: fix rdna4 mat_vec tuning (#29934) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52579133 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silico
CUDA: fix MMQ memory fault if n_expert >> n_ubatch (#29941) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52581427 macOS/iOS: macOS Apple Silicon (arm64)
cuda : move blocks_per_col to where it is used (#29939) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5
ci : set default permissions (#29945) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52595372 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (a
chat-peg-parser : clear current_tool when pending_tool_call is reset (#29942) A TOOL_ID node that arrives after TOOL_CLOSE wrote through current_tool, which still pointed into the just-destroyed pendi
ci : windows llvm build requires ninja multi-config (#29959) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52619546 macOS/iOS: macOS Apple Silicon (arm64)
cuda : move neu_padded to where it is used (#29940) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52636
ggml-cpu: support BF16/FP16/FP32 K tails in tinyBLAS on x86 (#29806) ggml-cpu: vectorize BF16 K tails in tinyBLAS tests: Skip tinyBLAS when use_ref is enabled so CPU tests compare against the vec_dot
metal : add tensor API flash attention kernel for F16 KV (#29570) metal : add tensor API flash attention kernel for F16 KV cont : add tensor FA kernels for DK=DV=512 and DK=576, DV=512 cont : support
model: support nimble decision model (#29844) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52332773 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Si
ggml-cpu : fix soft_max_back wrong output when dst aliases src1 (#27096) ggml-cpu : fix soft_max_back wrong output when dst aliases src1 GGML_OP_SOFT_MAX_BACK is listed in ggml_op_can_inplace, so the
ggml-quants : avoid invalid rounding in qkx3 scale search (#29817) ggml-quants : avoid invalid rounding in qkx3 scale search The imatrix scale search can produce an infinite, NaN, or otherwise out-of-
spec : add probabilistic sampling for simple draft and MTP (#27694) Make the drafter probabilistic and the target verify by rejection sampling Drop stale spec_draft_q before drafting Fallback to argma
CUDA: fuse shared experts into MMVQ (#29184) CUDA: fuse shared experts into MMVQ check if buffer is null move stride_col_dst to fusion args Website: https://llama.app Attestations: https://github.com/
model: add support for clef decision model (text-only) (#29831) init support for clef (text only) more static graph clean up nits nits 2 Update gguf-py/gguf/constants.py Co-authored-by: Sigbjørn Skjær
qwen4exp : halve the indexer score memory (#29825) qwen4exp : halve the indexer score memory The indexer scored all heads in one product and rectified a copy of it, so two [n_pool, n_idx_h, n_tokens]
ggml-openvino: update to 2026.4.1, optimize performance, expand ops, improve device listing. (#29852) ggml-openvino : Qwen3.5 MoE perf (#312) Squash of ravi9#312: ggml-openvino: add detailed inference
graph: gather the recurrent states once so the reserve covers every split (#29856) build_rs gathered the extra states (n_rs - n_seqs rows) with their own get_rows. The worst-case reserve has n_rs == n
ci: fix flaky ADD_ADD f16 by using the fused ADD tolerance (#29904) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52418791 macOS/iOS: macOS Apple Silicon
chat : honor json_schema in Ling 3.0 parser (#29813) chat : honor json_schema in Ling 3.0 parser Ling 3.0 only built a grammar for tool calls and did not handle inputs.json_schema, so response_format
common : add common_is_tty() helper and fix deprecated warnings on Windows (#29860) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-
server : fix laya abort by limiting n_batch to n_ubatch (#29903) server : fix laya abort by limiting n_batch to n_ubatch Fixes #29902 Assisted-by: Claude fix(review) : rm tests, embeddings cond Websit
vendor : update cpp-httplib to 0.59.0 (#29886) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52476117 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
mtmd : fix deprecated strdup warning on Windows (#29863) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/
Qwen4Exp: add MTP (#29761) Qwen4Exp: add MTP remove has_state member, check via ctx_bufs being non-empty consistent naming + less verbose comments cont : clean-up recurrent memory cont : clean-up comm
CUDA: Handle compute type for NVFP4 on cublass path (#29173) CUDA: Handle compute type for NVFP4 on cublass path Signed-off-by: ynankani ynankani@nvidia.com Use BF16 compute type for quantized models
llama : fix invalid assert in recurrent memory (#29799) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52014664 macOS/iOS: macOS Apple Silicon (arm64) macO
webgpu: add bfloat16 support for MUL_MAT/MUL_MAT_ID/GET_ROWS- #29358 (#29358) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52019658 macOS/iOS: macOS Appl
metal : release temporary private transfer buffers (#29777) metal : release temporary private transfer buffers Assisted-by: OpenAI Codex metal : fix order and formatting Co-authored-by: Niklas Wenzel
cuda : route sm70 to the Turing MMVQ nwarps table (#29753) cuda : route sm70 to the Turing MMVQ nwarps table Volta (sm_70) has no MMVQ parameter table of its own and falls through to GENERIC, which la
server: return HTTP 400 for invalid embedding requests (#29060) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52037093 macOS/iOS: macOS Apple Silicon (arm
hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA (#29685) hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA hex-cpy: various fixes on top of the co
llama : clamp kpool re-pool bound to existing pools (#29805) tests : simplify function signature llama : clamp kpool re-pool bound to existing pools The n_tokens/kpool + n_seqs_unq bound on n_new_g ov
common,rpc : fix cache dir creation through symlinks on buggy libstdc++ (#29816) See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101510 Close #29759 Signed-off-by: Adrien Gallouët angt@huggingface.co
CUDA: fix 2 broken Volta FA cases (#29803) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52093363 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
hexagon: add q2_k and q3_k quant type support (#29717) hexagon: add q2_k and q3_k quant type support hex-qk: consistent allocation of src1_row_size Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.co
qwen4exp: fix tests (#29819) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52169567 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, Kle
hexagon: install rebuilt HTP skels (#29828) hexagon: install rebuilt HTP skels Assisted-by: OpenCode hexagon: fix HTP skel catalog dependencies Assisted-by: OpenCode Website: https://llama.app Attesta
vulkan: add logging to pipeline compile issues (#29794) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52203370 macOS/iOS: macOS Apple Silicon (arm64) macO
ggml : add alloc_buffer_n to buffer type interface (#23671) ggml : add alloc_buffer_n to buffer type interface Add alloc_buffer_n method to ggml_backend_buffer_type_i interface, with a public API ggml
qwen4exp : optimize mask constructions (#29824) qwen4exp : optimize mask constructions cont : apply the same change for GLM5-next Website: https://llama.app Attestations: https://github.com/ggml-org/l
vulkan: disable large matmul tile on Samsung GPUs with 32KB shared memory (#28531) Assisted-by: Claude Opus Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/
llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) (#29818) init conversion convert: ok model loaded add server code improve conversion script support shared prompt prefix
llama : preserve original batch order for speculative decoding layer inputs (#29019) llama: preserve original batch order for layer inputs Assisted-by: Codex tests: cover layer-input order across KV l
args: fix cli download mmproj arg (#28977) tests: add tests for cli download arg parsing args: fix cli download mmproj arg Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cp
Hexagon: optimize ALLREDUCE with support for safe scatter mode (#29757) hex-allreduce: add support for safe scatter mode hex-allreduce: pare down excessive comments hex-allreduce: re-write to remove r
cuda: guard the iq4_nl dequantize row kernel against short rows (#29683) dequantize_block_iq4_nl writes QK_K values per block, but a row can be shorter than that (an IQ4_NL row is only guaranteed to b
ggml-opencl : replace alloca() with std::vector (#29765) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/
webgpu: fix SSM_SCAN binding aliasing (#29750) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51685637 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
model : re-enable -sm tensor for qwen4exp (#28569) #27941 disabled -sm tensor for qwen4exp because test-llama-archs asserted on the Meta device once the fixture carried a PLE layer: GGML_ASSERT(ggml_b
metal : use bf16 math for mxfp4 mul-mat (#29770) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51752863 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
llama-bench : fix verbosity filter to show GGML_LOG_ERROR (#28229) bench : fix verbosity filter to show GGML_LOG_ERROR (#28107) bench: remove dead variables Website: https://llama.app Attestations: ht
vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3 (#29734) vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3 The original tokenizer configs for PLaMo-2 and PLaMo-3 have add_bos_token: true a