b10612
tests : disable DOTS3NOTE arch test for WebGPU (#27654) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestation
Knowledge catalogue
tests : disable DOTS3NOTE arch test for WebGPU (#27654) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestation
metal: per-op source split + parallel compile (#26561) metal : per-op source split + parallel compile (#24021) preliminary extract common header op source split split metallib into 8 libs && load in p
metal : per-device tuned (Q, NE) for flash-attn vec (#26570) metal : per-device tuned (Q, NE) for flash-attn vec (#25750) rebase Q-generic FA vec body from 01dc936 (#23114) add 53 f16 (Q,NE) flash-att
cuda : add POOL_1D support (#27573) cuda : add POOL_1D support fix: add missing trailing newline for editorconfig compliance Website: https://llama.app Attestations: https://github.com/ggml-org/llama.
vendor : update subprocess.h (#27409) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42402532 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (a
DeepseekV4: fix rollback with multi-seq (#26756) DeepseekV4: fix rollback with multi-seq fix model loading make pending rollback single use only clear cache for seq_id for full load add assert for com
common : skip device_info loop if it's not going to be printed (#26692) The device_info loop iterates over the discovered devices and gets the available and total memory counts. With the CUDA backend
server : add LLAMA_SERVER_SLOTS_N_DIFF (#27600) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42423433 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
mtmd: use pillow-accurate algo, correct resize_algo for all models (#27594) mtmd: use pillow-accurate resize algo, correct resize_algo for all models speed optimization Website: https://llama.app Atte
test: move tools/parser to tests (#27548) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42442638 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silico
model : support MTP in GLM-4.5-Air (#26534) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42447665 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sili
model: add dots3-note (#27060) text: conversion init impl address review comments fix rope move to a new llama_kv_cache_dsa_iswa Website: https://llama.app Attestations: https://github.com/ggml-org/ll
mtmd: support webp via ffmpeg (#27520) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42289035 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (
sycl : add Q2_K reordered MMVQ and ESIMD kernels (again) (#27490) Revert 'Revert 'sycl : add Q2_K reordered MMVQ and ESIMD kernels (#26336)' (#…' This reverts commit 7a0e42f. add gate params Website:
common : fix draft-mtp with embeddings (#26352, #27299) (#27400) common: fix draft-mtp with embeddings (#26352) --whitespace Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Website:
ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (#24575) ggml: optimize concat op by replacing per-element memcpy with row-level memcpy ggml: fix concat offsets for row-
mtmd: support dots3-note vision+audio (#27524) text: conversion init impl mtmd: conversion impl mtmd cpp Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggin
model : support DSpark for bailingmoe3 (#27508) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42309942 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
ci : Restore ROCm job for Ubuntu (#27399) Revert 'ci : disable ubuntu-rocm (#26969)' This reverts commit 9558fa4. ci: set ccache compiler_check=content for ROCm build The ROCm toolchain is pip-install
fit: also take into account n_streams (#27496) fit: also take into account n_streams server: make the draft context follow the target context With a non-unified KV cache the target context now holds n
common: add json.h abstraction (#27511) add common/json migrate common adapt jinja migrate server big wip migrate tests wip revert some excessive changes wip wip 2 revert redundant changes fix server
mtmd: use ggml_rope_set_offset (#27521) mtmd: use ggml_rope_set_offset add comment Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42334609 macOS/iOS: macOS
vulkan : added the PAD_REFLECT_1D operation (#26586) vulkan : added PAD_REFLECT_1D operation Implemented the GGML_OP_PAD_REFLECT_1D operation for the Vulkan backend Changes: pad_reflect_1d.comp: imple
common: json.h: fix clang lto (#27575) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42368523 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (
Revert 'tensor-split meta backend fixes (#26502)' (#27433) This reverts commit d59d455. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42012577 macOS/iOS:
metal : dequantize quantized KV to F16 before flash attention (#27390) metal: dequantize q8_0 KV to f16 before flash attention Add a preprocessing pass for GGML_OP_FLASH_ATTN_EXT on the Metal backend:
common : gracefully fallback on unsupported regex patterns in JSON schema (#26939) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42018860 macOS/iOS: macOS
CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover (#26079) CUDA: runtime GGML_CUDA_MMVQ_MAX to tune the mvq->MMQ decode crossover Add a runtime override of the mul
server: (router) lazy-load startup_models after main setup (#27424) server: (router) lazy-load startup_models after main setup only allow is_first_load to populate it nits nits 2 Website: https://llam
CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows (#26678) CI: Use LLVM's OpenMP over MSFT_DEBUG_non_redist on Windows Currently, we ship the non-redist debug version of microsoft's libomp.
metal : dequant kv cache only for large batches (#27438) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42031082 macOS/iOS: macOS Apple Silicon (arm64) mac
vulkan: FA MMQ should use fp32 for Q quantization calculations (#27413) Codex found that qd could be a denorm and 1/qd would overflow. Website: https://llama.app Attestations: https://github.com/ggml-
mtmd: add --mmproj-device argument (#23255) feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var feat: load mmproj device backend immediately, add -mmdev shortflag fix: its
metal : clamp K extent in tensor API mat-mat kernel for K not a multiple of 32 (#27450) The Tensor API mat-mat path of kernel_mul_mm (GGML_METAL_HAS_TENSOR) fed a static K=32 tile to the matmul2d op o
TP: enable tensor split for LFM2/LFM2MOE (#26993) Assisted-by: deepseek-v4-flash Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42096995 macOS/iOS: macOS A
llama.cpp : bump version to 0.2.0 (#27498) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42207505 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
ci : run ccache-clear as the last step of release jobs (#27503) ci : run ccache-clear as the last step of release jobs Assisted-by: pi:llama.cpp/Qwen3.8-27B update disabled job too to force rebase Co-
model: use ggml_rope_set_offset() (#27382) model: use ggml_rope_set_offset() partially apply to deepseek2 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42
Overview New version has been released. Nightly build: b10566 Web UI: the nightly-tag.txt asset contains the tag of the corresponding nightly release More info: dist : releases and versioning of ggml-
server: add dedup-cache-models preset option (#27346) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCF
metal : dequantize q8_0 using packed types (#27370) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFra
mtmd: add mtmd_bitmap_set_mergeable (#27348) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework
ggml: add ggml_rope_set_offset (+ metal support) (#27120) add params cpu kernel metal kernel add test backend ops gate other backends ggml: (cuda) support ggml_rope_set_offset (#27121) rm cuda support
model : GraniteSWAForCausalLM / GraniteMoeSWAForCausalLM (#25505) feat(convert): Add conversion for GraniteSWAForCausalLM Branch: GraniteSWAForCausalLM AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Sig
vulkan: add null checks in ggml_vk_queue_command_pools_cleanup (#27353) Guard against null queue pointers. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/4
vulkan : dequant q8_0 KV once in coopmat1 (#25494) vulkan : dequant q8_0 KV once in coopmat1 Assisted-by: Claude (Opus 4.8) vulkan : fall back instead of aborting when FA scratch exceeds maxStorageBuf
server: refactor sleep handling, allow access /metrics during sleep (#27376) add cached responses refactor on_sleeping_state allow accessing metrics during sleep metrics task should not reset timer up
ggml-cpu: gate __fp16 on __ARM_FP16_FORMAT_IEEE (#26860) ggml-cpu: gate __fp16 on __ARM_FP16_FORMAT_IEEE __ARM_NEON only signals NEON availability. The __fp16 type also needs the IEEE half format, imp
ci : add attestation for signed release artifacts (#25933) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/41614541 macOS/iOS: macOS Apple Silicon (arm64) m
CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark Signed-off-by: ynankani ynankani@nvidia.com skip moe experts and allow others based on k geometry (allow only small idle tail) Signed-off-by:
build : fix xcframework + cmake clean-up (#27304) xcframework : fix build mtmd : remove unused include path vendor : use vendor::hash alias target in cmake CMake reserves '::' in target names for impo
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu ar
mtmd: fix LFM2 image tiling threshold (#27057) mtmd: fix LFM2 image tiling threshold refactor testing fix fix on windows Co-authored-by: Xuan Son Nguyen son@huggingface.co Website: https://llama.app m
ci : Update OpenVINO to 2026.3, skip nemotron-h rollback test (#27292) update to ov-2026.3, update device drivers ci: skip nemotron-h rollback test on OpenVINO The OpenVINO backend does not support SS
[SYCL] support OP OPT_STEP_ADAMW, OPT_STEP_SGD (#25268) fix conflict fix conflict of ops.md fix conflict of ops.md update the ops.md Co-authored-by: Neo Zhang Jianyu jianyu.zhang@intel.com Website: ht
sycl: fix thread/block count in quantized cpy kernel launches (#27160) Adjusts the thread/block count to be proportional to the size of the quant, reducing under/over subscription. Largest perf improv
ci : push release tag explicitly in release.yml (#27261) Add a 'Create and push git tag' step to the release job, right before the 'Create release' step. The tag is created with git tag and pushed wit
cuda : skip UMA override for HIP builds (#27083) AMD APUs report accurate memory via hipMemGetInfo. Using MemAvailable over-promises on small-carveout systems. fixes #18159 Website: https://llama.app
llama : check LoRA tensor data is within file bounds (#27056) llama : check LoRA tensor data is within file bounds Update src/llama-adapter.cpp Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggi
chat: refactor handling supports_string_content / supports_typed_content (#27130) better supports_string_content cap detect test: add 'skip' messages_inp_normalizer Website: https://llama.app macOS/iO