Search: “llama-cpp-releases”

ggml-zdnn : introduce Q8_0 mat mul (#29836) ggml-zdnn: initial prototype of q8_0 matmul Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: update zdnn docs Signed-off-by: Aaron Teo aaron.teo1@ibm.com S

graph : avoid reallocations in the 'no cache' path (#30439) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55354972 macOS/iOS: macOS Apple Silicon (arm64)

cuda : warnings cleanup Signed-off-by: Adrien Gallouët angt@huggingface.co Update ggml/src/ggml-cuda/common.cuh Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Signed-off-by: Adrien

cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (#30327) ggml-cuda: gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG ggml-cuda: cache env var lookups in static const instead of re-readi

vulkan: allow multiple devices to access gpu-visible host memory (#29741) vulkan: allow multiple devices to access gpu-visible host memory clean up erroneous newlines add Windows support handle nullpt

tests : consolidate context state tests into test-llama-context (#30023) tests : consolidate the context state tests into test-llama-context test-save-load-state, test-recurrent-state-rollback and tes

graph: skip the LM head for pooled embeddings (#30445) graph: skip the LM head for pooled embeddings A context that extracts pooled embeddings outputs one vector per sequence, but every model still ex

model: load YaRN beta metadata for laguna models (#30331) Assisted-by: Codex Co-authored-by: Adam Caldwell accaldwell@users.noreply.github.com Website: https://llama.app Attestations: https://github.c

llama : do not map foreign ranges into a backend buffer A buffer created from a host pointer pins everything that it wraps. When several contexts share the same file (lazy tensors, MoE tensors kept on

hexagon: add IM2COL_3D (#30305) hexagon: add IM2COL_3D support Assisted-by: OpenCode hexagon: add DMA paths for IM2COL_3D Assisted-by: OpenCode hexagon: avoid scalar fp16 conversions in IM2COL Assiste

vulkan: fix buffer logging on 32-bit platforms after source split (#30326) Move the vk::Buffer stream operator to the shared header so it is visible to ggml-vulkan-debug.cpp. Website: https://llama.ap

ggml: Change default visibility of symbols to hidden (#30014) See discussion in #29997. This prevents a handful of internal symbols leaking via GGML_API. Website: https://llama.app Attestations: https

vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size (#30164) vulkan: specialize quant block size for CM2 matmul vulkan: use ternary expressions to avoid switch lowering f

sycl: guard DMMV reads at row tails (#29338) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54988744 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sil

llama : report compute graph reallocations during inference (#30329) Add an optional ggml_backend_sched_realloc_callback that user code registers with ggml_backend_sched_set_realloc_callback to be not

metal : fix mul_mat_id with duplicate ids (#30373) metal : fix mul_mat_id limits and kernel name metal : fix mul_mat_id with duplicate ids kernel_mul_mm_id_map0 assumed at most one match per token and

tests : link each test against only the libraries it uses (#30299) Give llama_build() and llama_build_and_test() a LINK keyword (default: llama-common) so call sites can declare what a test actually n

model: support vllm decision 2 model (#30378) models: support pplx-decider vllm d2 update clarify naming rule stricter validation fix ci rm redundant filter_tensors Website: https://llama.app Attestat

model: support vllm d3 models (#30383) models: support pplx-decider vllm d2 update clarify naming rule stricter validation d3 Website: https://llama.app Attestations: https://github.com/ggml-org/llama

vulkan: pack FMA for f16 accumulation, no dot product op (#29877) vulkan: pack FMA for f16 accumulation, no dot product op vulkan: dot_product packs two rows per f16 FMA vulkan: rename dot_product to

metal: fix metal tensor kernel_mul_mm error for large tensor (#28748) metal: fix metal tensor kernel_mul_mm error for large tensor Assisted-by: Codex Update ggml/src/ggml-metal/kernels/mul_mm.metal Co

The no-cache path builds a [n_tokens, n_tokens] mask and passes K and V unpadded, so FLASH_ATTN_EXT sees K->ne[1] == n_tokens. The CUDA kernels for head size 512 require a multiple of FATTN_KQ_STRIDE,

model: fix vllm d3 multi seq case (#30408) model: fix vllm d3 multi seq case add TODO Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55219202 macOS/iOS: ma

server: add component tests (#30312) server: add component tests nits add codeowners Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54782864 macOS/iOS: mac

vulkan: handle mul_mat_id duplicates in the prepass rather than looping (#29998) vulkan: compute every row of an expert in mul_mm_id when ids repeat vulkan: handle mul_mat_id duplicates in the prepass

cpu: fix AVX2 CPUs not using F16C when available (#30270) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54796975 macOS/iOS: macOS Apple Silicon (arm64) ma

model : support for Prism Bonsai 2 27B (#29600) Runtime support for Prism Bonsai 2 27B Assisted-by: Claude Code address prism hadamard runtime feedback move hadamard tensors into method, define folded

rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs GGML_RPC_DEBUG is now parsed as a number: 0/unset disables debug logs, 1-3 emit increasingly detailed output (events, per-command trace, t

opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rms_norm (#30310) opencl: fallback when Q4_K weight images exceed device limits Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com

opencl: improve fa, allow dk512 for gemma-4, improve dk64 (#30266) opencl: enable Gemma-4 E4B GPU decode Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com opencl: extend Gemma-4 GPU decode to E2B

sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering (#29809) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54540406 macOS/iOS: macO

opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin (#30187) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5457

cmake(s390x): disable z17 target for unsupported compilers (#30297) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54655455 macOS/iOS: macOS Apple Silicon

model : support MiniCPM-V 4.7 (#29416) mtmd : add MiniCPM-V 4.7 support Signed-off-by: tc-mb tianchi_cai@icloud.com model : allow mrope time from an extra position slot Signed-off-by: tc-mb tianchi_ca

server: leave a busy slot untouched when a request pins it (#30295) A request asking for a busy id_slot still ran the prompt cache update on that slot before being deferred. When the RAM cache held a

hexagon: fix IM2COL patch-embed DMA ring overflow (#30189) hexagon: fix IM2COL patch-embed DMA ring overflow The exact-tiling (stride == kernel, no pad/dilation) IM2COL DMA kernel issues ICKH DDR->VTC

CUDA: pass src1 precision to host MMQ config helpers (#30168) CUDA: pass src1 precision to host MMQ config helpers Signed-off-by: ynankani ynankani@nvidia.com make src1 prec explicit Signed-off-by: yn

metal: add the 128/96 flash attention kernels, accept every compiled pair (#30209) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54260937 macOS/iOS: macOS

server: use port 9931 by default (#30159) server: use port 9931 by default revert unrelated changes Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54269123

ci: fix Models Backend webgpu by supporting GGML_OP_DUP (#30216) The mixed batch path of PR-29622 writes the token rows with set_rows into a dup of the embeddings. WebGPU did not support DUP, so the d

musa: drop mp_21 from the default architectures (#30203) musa: drop mp_21 from the default architectures Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Address review comments Signed-off-by: Xiao

vulkan: fix rms_norm workgroup count overflow (#30145) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54337638 macOS/iOS: macOS Apple Silicon (arm64) macOS

webgpu: use 2D workgroup dispatch for all the ops which use 1D dispatch (e.g., rms_norm) (#30219) remove 1D workgroups dispatching formatting Website: https://llama.app Attestations: https://github.co

meta : handle host views (#30217) meta : handle views of tensors allocated on the host A view shares the memory of its view_src, so ggml-alloc never allocates a view in the buffer of the split it land

ggml: fix OOB write in ggml_acc with negative offset (#30135) ggml_acc_impl narrowed a size_t offset to int32_t without checking that it fits, so a large offset could truncate to a negative int32_t. T

llama: keep the backend sampling graph static across ubatches (#30223) The reserve builds n_outputs_max_per_seq sampling chains per sampler, while a decode built one per output row, so the graph chang