b11019
gguf : align the data section relative to the GGUF start, not the file (#28993) gguf : align the data section relative to the GGUF start, not the file gguf_init_from_file_ptr reads a GGUF from the cur
Knowledge catalogue
gguf : align the data section relative to the GGUF start, not the file (#28993) gguf : align the data section relative to the GGUF start, not the file gguf_init_from_file_ptr reads a GGUF from the cur
chat : add message delimiters to the DeepSeek V3.2/V4 parser (#29008) chat : add message delimiters to the DeepSeek V3.2/V4 parser Assisted-by: Claude Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaere
vulkan: split buffers and debug code into separate files, add shared headers (#28732) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48170206 macOS/iOS: ma
openvino : Update OpenVINO to 2026.4;fix clangd,MSVC warnings; (#29009) Update to openvino-2026.4 Update OV docs ggml-openvino : fix clangd and MSVC warnings fix int to ptr cast, more internal linkage
model : extend Nemotron MTP support (#29018) first fix removed unnecessary declarations Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48188464 macOS/iOS:
model : skip gate_up_exps if TENSOR_SKIP is set (#29014) required for qwen35moe if MTP tensors are fused but not loaded Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/a
Hey all, Henry from Cactus Compute here, I kinda wanted to share our latest model and get feedback from the family :) Needle 3 is a small foundation model for automation: you give it the functions you
ci : add self-hosted webgpu to hf-jobs (#28712) add self-hosted vulkan and webgpu to hf-jobs try t4-medium cont : adjust cpu backend threads try t4-small again restore cm jobs Co-authored-by: Georgi G
vulkan: make MUL_MAT_ID BN/2 tail unconditional (#28923) Use BN/2 as the default for BNover2 and as the disabled fallback for BNover4, and remove the enable gate from the MUL_MAT_ID BN/2 branch. The B
HIP: broaden MoE ncols_opt tile heuristic on RDNA3.5 architecture (#28935) It's found the MoE ncols_opt tile heuristic needs to be broadened to include the RDNA3.5 architecture. The code change is imp
rpc : invalidate cached compute graph when a referenced buffer is freed (#24292) The server caches the most recent compute graph per device so that GRAPH_RECOMPUTE can re-execute it without resending
spacemit : fix wrong transpose function for int16 data (#25161) The sizeof(int16_t) branch in permute_transpose_impl calls rvv_transposed_s32_mn_to_nm instead of rvv_transposed_s16_mn_to_nm. This is a
model : add support for HrmTextForCausalLM (DFM Mimir 1B) (#27625) model : add support for HrmTextForCausalLM (DFM Mimir 1B) HRM-Text runs two transformer stacks (low, high) in an alternating cycle ov
hexagon: accept the zeroed rope probe in supports_op (#28995) llama probes weight placement with a rope where all params are 0, so rejecting n_dims == 0 or freq_base == 0 puts rope_freqs on the CPU. T
hexagon: Support for K-Quants Q4_K and Q6_K (#28994) implement q6k/q4k kernels Squashed from: feat: implement q6k kernel hex-q6k: improve unpack accuracy hex-q4_k: add support for Q4_K kernels Co-auth
Enable CUDA graph for MTP draft (#28549) Improve CUDA graph usage for MTP Rename field Address review feedback Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestatio
TP: fix split state and granularity for fused QKV gemma4, qwen35 (#28965) model: calculate split states for attn_qkv from n_head * n_embd_head_k required for gemma4 with --fuse-qkv, where n_embd is 53
Confidential Computing keeps your data from being tracked or used in three ways: - End-to-end encrypted inference - Hardware-enforced isolation that extends to the GPU with @nvidia Confidential Comput
arXiv:2609.13846v1 Announce Type: cross Abstract: Delayed Tensor Parallelism (DTP) removes the blocking all-reduce of tensor-parallel Transformer inference. Every device adds its own partial output to
metal : add FA kernels for HSK=96, HSV=64 (MiniCPM3) (#28599) metal : add FA kernels for HSK=96, HSV=64 (MiniCPM3) MiniCPM3 sets attention.key_length to 96 and does not set attention.value_length, whi
opencl: add generic ssm_scan (#28881) opencl: add generic ssm_scan opencl: fix whitespace Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47601188 macOS/iOS
OpenVINO: optimize stateful decode and GPU MoE inference (#28638) exclude GPU/NPU failing POOL_2D case Fix pool case ggml-openvino: fix stateful decode for Gemma-4 per-layer-type head sizes ggml-openv
vulkan: support sparse Flash Attention (#28105) vulkan: add sparse Flash Attention support for DSV4/GLM tune implementation add tests avoid nondeterministic atomicAdd add cm2 decode vector support sim
models : move build_arch_graph() after graph() template specialization (#28934) Move build_arch_graph()'s function definitions after the graph and graph template specializations have been explicitly d
cuda: support row-contiguous SUM_ROWS (#26308) cuda: support row-contiguous SUM_ROWS organize the code and add GGML_OP_MEAN to support row-contiguous tensors using the same shared kernel, and add a te
rpc : hash-cache only weights (#28789) rpc : hash-cache only weights ggml_backend_rpc_buffer_set_tensor and ggml_backend_rpc_set_tensor_async hashed every transfer above HASH_THRESHOLD and let rpc-ser
opencl: choose the MoE expert matmul by batch size for speculative decoding/MTP (#27637) opencl: gate the prebuilt q4_0 MoE GEMM on routing count opencl: stop writing zeros into the padded MoE activat
hexagon: add back missing contiguous fast-path and hvx_copy_uu for each run (#28886) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47775478 macOS/iOS: mac
arXiv:2609.13260v1 Announce Type: cross Abstract: This paper proposes SpecAugment-Patch Merging, a simple yet effective method to accelerate Audio Spectrogram Transformer (AST) training. We first appl
arXiv:2609.13551v1 Announce Type: new Abstract: Static-image benchmarks do not capture the computational and temporal requirements of practical orchard video analytics. This study presents an end-to-e
ggml-cuda: fallback to F32 on device without BF16 hardware acceleration (#28846) ggml-cuda: fallback to F32 on device without BF16 hardware acceleration: (Nvidia >= AMPERE, AMD >= RDNA3 or = CDNA) app
common : move llama_n_rs_seq to before llama_decode (#28749) This commit moves the llama_n_rs_seq function call to before the llama_decode call and returns directly if the check is true, removing the
ggml-cpu : disable PCH and fix CACHE_LINE_SIZE ambiguity to fix heap corruption (#28882) Disable the ggml-cpu precompiled header and remove the std::hardware_destructive_interference_size branch from
sycl: rfc: Use radix select for top_k (#28670) sycl: GPU-resident TOP_K for large k, parallelised over the device The SYCL backend refused GGML_OP_TOP_K above k = 32 and let it fall back to the CPU, a
ci : add ubuntu-cuda builds to release (#28186) release : add ubuntu-cuda build job (12.8/13.3, x64+arm64) Add GCC 14 for CUDA arm64 builds in CI Eplicit bash Install git for CCCL fetch Install git be
HIP: fattn-mma: use fp32 accumulation on MFMA devices (#28576) use fp32 accumulators in fattn-mma on CDNA Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47
I feel like the data sovereignty movement for firms is going to run headlong into the challenge that maintaining fine-tuned small models in a way that continually beats larger general-purpose models i
common : implement common_schema internal representation for JSON schemas (#28736) common : implement common_schema types common : implement a json schema optimizer common : reduce optimizations commo
common: add LOG_JSON macro to log structured data (#28586) add LOG_JSON macro fit: add demo LOG_JSON Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/4711287
chat : improve parsing of complex types in qwen3-coder (#28742) chat : improve schema support in qwen3 parser cont : clean up grammar a bit Website: https://llama.app Attestations: https://github.com/
opencl: apply the noshuffle row-alignment rule to q4_K, q5_K and q8_0, not just q6_K (#28575) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47132459 macOS
vulkan: workaround NV queuesubmit driver bug (#28830) There is a driver bug where two queues on the same VkDevice simultaneously submitting can break some internal synchronization. Until it's fixed, a
sycl : Fix get mem error (#28227) fix for unsupport zes API optimize the code adjust the log level rm unused head files Update docs/backend/SYCL.md Co-authored-by: Titaniumtown titaniumtown@proton.me
models : guard the expert FFN size fallback in nemotron-h against a zero divisor (#28779) The NextN/MTP tail loop derives the expert FFN size as n_ff/n_expert_used when expert_feed_forward_length give
tests : exclude HY_V4 from WebGPU test-llama-archs tests (#28855) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/a
I think once remote connections to “your” Claude or Astra or Claw or whatever become easier and more common, on-phone assistants like Siri will lose a lot of value. You want one smart AI managing ever
ggml-webgpu: Update to a recent version of Dawn (#28683) ggml-webgpu: Update to a recent version of Dawn No module scanning Accept review suggestion to update comment Co-authored-by: Masashi Yoshimura
hexagon: support for multi-device model split (aka row-split) (#28589) hex-row-split: add support for multi-device row spliting Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com hex-mdev: add work
webgpu: align tensor bindings to the type block size (#28382) Walk the binding offset back until the distance to the tensor is a whole number of blocks, so block quantized views get a valid element of
opencl: add bin kernel kernel_gemm_noshuffle_q4_k_f32_32b_trans_ila_a8_bin (#28677) opencl: add A8 Q4_K non-MoE binary kernel opencl: fix layout compatibility opencl: rename binary kernel selection he
server: frame the router child state command as a whole line (#28747) The child writes its state commands on stdout while the logger writes on stderr, and both share a single pipe. The logger emits th
cmake: leave the timestamp out of precompiled headers on clang (#28816) Clang stores the modification time of the precompiled header sources inside the header and refuses the header when they differ.
vulkan: fix data race and OOB access in argsort(large) (#28705) argsort had a data race in the inner loop, which VVL caught. But I don't think this was causing failures in practice. argsort_large has
CUDA/HIP: Flash Attention tuning (gfx1201) (#28102) HIP: enable mma FA for head size 256 on RDNA4, tune configs Assisted-by: Claude Assisted-by: Codex HIP: prefer whole-tile FA grids over stream-k on
server: fix speculation after an image (#28715) server: fix speculation after an image Pass the actual position to the drafter after an image, instead of the token count. Affects every drafter, not ju
model : fix MTP context kv cache allocation for deepseek2, glm4moe, … (#28630) model : fix MTP context kv cache allocation for deepseek2, glm4moe, cohere2moe architectures (#28626) model: add inverse
metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 Generalize the row split from #28086 to
metal : single-source fusion table + fusion debug rework (#28164) metal : rework fusion patterns into a single table All fusable op patterns for the Metal backend are now declared once in a fusion tab
cmake : skip PCH for llama-server PCH when using MSVC (#28763) This commit fixes an issue that I introduced when adding PCH (precompiled headers) in Commit 3bcfeb7 ('cmake : add PCH and unity build to
model: fix all granite family parameter counts (#28643) model: fix all granite family parameter counts Signed-off-by: Aaron Teo aaron.teo1@ibm.com model: fix additional include, add missing A prefix f