All entries

rpc: support apple RDMA as an RPC transport (#26421) rpc: support apple RDMA as an RPC transport remove set_tensor micro optimization, rpc socket pinning per CR remove transparent reconnect trigger ap

ci : update OS used for ROCM to Ubuntu 24.04 (#27681) This matches what other build targets use and also what AMD advertises wheels as supporting. Website: https://llama.app Attestations: https://gith

kleidiai: Rework KleidiAI Build System/Integration (#26077) Rework KleidiAI Build System/Integration Signed-off-by: Jonathan Clohessy Jonathan.Clohessy@arm.com Add fp16 guard, and fix cmake caching is

Mariko Wakabayashi is a Principal Applied Scientist at Microsoft who leads the development of agentic AI workflows for cybersecurity operations. She specializes in evaluating LLM‑powered systems and a

Overview llama.cpp 0.3.0 introduces the dots3-note multimodal model (with a new DSA-ISWA KV cache), MTP support for GLM-4.5-Air, and tensor-split (-sm tensor) plus multi-sequence rollback fixes for De

Deepseek 4: -sm tensor (#26490) DSV4: sm tensor set coarser granularity for head splits fix dspark add model saving for dsv4 + allow dflash to return on specific device add comment about dsv4 seq_rm s

mamba2 : Flatten in/out projections to dispatch GEMM instead of GEMV (#27513) mamba2 : flatten mamba2 in/out projections to dispatch gemm instead of gemv mamba2 : remove redundant output reshape Websi

ggml : fix ggml_clamp (#27644) ggml : fix ggml_clamp cont : update ggml-alloc Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42576162 macOS/iOS: macOS Appl

mtmd: video: fix moov atom at the end of file (#27596) mtmd: video: fix moov at the end of file Co-authored-by: rkfg rkfg@rkfg.me fix SIGPIPE windows: handle broken pipe case Co-authored-by: rkfg rkfg

ggml : shorten virtual device naming in CUDA and Metal (#27608) ggml : shorten virtual device naming in CUDA and Metal Assisted-by: llama.cpp:DeepSeek-V4-Flash-0731 ggml-metal : build device descripti

tests : disable DOTS3NOTE arch test for WebGPU (#27654) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestation

metal: per-op source split + parallel compile (#26561) metal : per-op source split + parallel compile (#24021) preliminary extract common header op source split split metallib into 8 libs && load in p

metal : per-device tuned (Q, NE) for flash-attn vec (#26570) metal : per-device tuned (Q, NE) for flash-attn vec (#25750) rebase Q-generic FA vec body from 01dc936 (#23114) add 53 f16 (Q,NE) flash-att

cuda : add POOL_1D support (#27573) cuda : add POOL_1D support fix: add missing trailing newline for editorconfig compliance Website: https://llama.app Attestations: https://github.com/ggml-org/llama.

vendor : update subprocess.h (#27409) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42402532 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (a

DeepseekV4: fix rollback with multi-seq (#26756) DeepseekV4: fix rollback with multi-seq fix model loading make pending rollback single use only clear cache for seq_id for full load add assert for com

common : skip device_info loop if it's not going to be printed (#26692) The device_info loop iterates over the discovered devices and gets the available and total memory counts. With the CUDA backend

server : add LLAMA_SERVER_SLOTS_N_DIFF (#27600) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42423433 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple

mtmd: use pillow-accurate algo, correct resize_algo for all models (#27594) mtmd: use pillow-accurate resize algo, correct resize_algo for all models speed optimization Website: https://llama.app Atte

test: move tools/parser to tests (#27548) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42442638 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silico

model : support MTP in GLM-4.5-Air (#26534) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42447665 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sili

model: add dots3-note (#27060) text: conversion init impl address review comments fix rope move to a new llama_kv_cache_dsa_iswa Website: https://llama.app Attestations: https://github.com/ggml-org/ll

mtmd: support webp via ffmpeg (#27520) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42289035 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (

sycl : add Q2_K reordered MMVQ and ESIMD kernels (again) (#27490) Revert 'Revert 'sycl : add Q2_K reordered MMVQ and ESIMD kernels (#26336)' (#…' This reverts commit 7a0e42f. add gate params Website:

common : fix draft-mtp with embeddings (#26352, #27299) (#27400) common: fix draft-mtp with embeddings (#26352) --whitespace Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Website:

ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (#24575) ggml: optimize concat op by replacing per-element memcpy with row-level memcpy ggml: fix concat offsets for row-

mtmd: support dots3-note vision+audio (#27524) text: conversion init impl mtmd: conversion impl mtmd cpp Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggin

model : support DSpark for bailingmoe3 (#27508) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42309942 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple

ci : Restore ROCm job for Ubuntu (#27399) Revert 'ci : disable ubuntu-rocm (#26969)' This reverts commit 9558fa4. ci: set ccache compiler_check=content for ROCm build The ROCm toolchain is pip-install

fit: also take into account n_streams (#27496) fit: also take into account n_streams server: make the draft context follow the target context With a non-unified KV cache the target context now holds n

common: add json.h abstraction (#27511) add common/json migrate common adapt jinja migrate server big wip migrate tests wip revert some excessive changes wip wip 2 revert redundant changes fix server

mtmd: use ggml_rope_set_offset (#27521) mtmd: use ggml_rope_set_offset add comment Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42334609 macOS/iOS: macOS

vulkan : added the PAD_REFLECT_1D operation (#26586) vulkan : added PAD_REFLECT_1D operation Implemented the GGML_OP_PAD_REFLECT_1D operation for the Vulkan backend Changes: pad_reflect_1d.comp: imple

common: json.h: fix clang lto (#27575) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42368523 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (

What's Changed mlx: fix mac assumptions on linux/windows by @dhiltgen in #17898 mlx update by @dhiltgen in #17886 lint fixes by @dhiltgen in #17897 app: add claude desktop app by @ParthSareen in #1789

Revert 'tensor-split meta backend fixes (#26502)' (#27433) This reverts commit d59d455. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42012577 macOS/iOS:

metal : dequantize quantized KV to F16 before flash attention (#27390) metal: dequantize q8_0 KV to f16 before flash attention Add a preprocessing pass for GGML_OP_FLASH_ATTN_EXT on the Metal backend:

common : gracefully fallback on unsupported regex patterns in JSON schema (#26939) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42018860 macOS/iOS: macOS

CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover (#26079) CUDA: runtime GGML_CUDA_MMVQ_MAX to tune the mvq->MMQ decode crossover Add a runtime override of the mul

server: (router) lazy-load startup_models after main setup (#27424) server: (router) lazy-load startup_models after main setup only allow is_first_load to populate it nits nits 2 Website: https://llam

CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows (#26678) CI: Use LLVM's OpenMP over MSFT_DEBUG_non_redist on Windows Currently, we ship the non-redist debug version of microsoft's libomp.

metal : dequant kv cache only for large batches (#27438) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42031082 macOS/iOS: macOS Apple Silicon (arm64) mac

vulkan: FA MMQ should use fp32 for Q quantization calculations (#27413) Codex found that qd could be a denorm and 1/qd would overflow. Website: https://llama.app Attestations: https://github.com/ggml-

mtmd: add --mmproj-device argument (#23255) feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var feat: load mmproj device backend immediately, add -mmdev shortflag fix: its

metal : clamp K extent in tensor API mat-mat kernel for K not a multiple of 32 (#27450) The Tensor API mat-mat path of kernel_mul_mm (GGML_METAL_HAS_TENSOR) fed a static K=32 tile to the matmul2d op o

TP: enable tensor split for LFM2/LFM2MOE (#26993) Assisted-by: deepseek-v4-flash Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42096995 macOS/iOS: macOS A