All entries

hexagon: support for multi-device model split (aka row-split) (#28589) hex-row-split: add support for multi-device row spliting Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com hex-mdev: add work

webgpu: align tensor bindings to the type block size (#28382) Walk the binding offset back until the distance to the tensor is a whole number of blocks, so block quantized views get a valid element of

opencl: add bin kernel kernel_gemm_noshuffle_q4_k_f32_32b_trans_ila_a8_bin (#28677) opencl: add A8 Q4_K non-MoE binary kernel opencl: fix layout compatibility opencl: rename binary kernel selection he

opencl: fix several bugs where the backend aborts (#27630) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47037447 macOS/iOS: macOS Apple Silicon (arm64) m

server: frame the router child state command as a whole line (#28747) The child writes its state commands on stdout while the logger writes on stderr, and both share a single pipe. The logger emits th

syscl : Handle (fail gracefully) unsupported tq1_0 quants (#28681) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47043623 macOS/iOS: macOS Apple Silicon (

vendor : update cpp-httplib to 0.56.0 (#28787) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47045738 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S

ggml-cuda: hip add specific config table for AMD GCN (#27841) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47052867 macOS/iOS: macOS Apple Silicon (arm64

server : allow model downloads at model limit fix issue #26809 (#28530) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47055500 macOS/iOS: macOS Apple Sili

ui : add cache (#28802) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47065031 macOS/iOS: macOS Apple S

cmake: leave the timestamp out of precompiled headers on clang (#28816) Clang stores the modification time of the precompiled header sources inside the header and refuses the header when they differ.

jinja : support dot property integer literals (#28817) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47107718 macOS/iOS: macOS Apple Silicon (arm64) macOS

opencl: add A8 Q4_0 mm binary kernel support (#28268) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46774832 macOS/iOS: macOS Apple Silicon (arm64) macOS

vulkan: fix data race and OOB access in argsort(large) (#28705) argsort had a data race in the inner loop, which VVL caught. But I don't think this was causing failures in practice. argsort_large has

CUDA/HIP: Flash Attention tuning (gfx1201) (#28102) HIP: enable mma FA for head size 256 on RDNA4, tune configs Assisted-by: Claude Assisted-by: Codex HIP: prefer whole-tile FA grids over stream-k on

server: fix speculation after an image (#28715) server: fix speculation after an image Pass the actual position to the drafter after an image, instead of the token count. Affects every drafter, not ju

model : fix MTP context kv cache allocation for deepseek2, glm4moe, … (#28630) model : fix MTP context kv cache allocation for deepseek2, glm4moe, cohere2moe architectures (#28626) model: add inverse

metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 Generalize the row split from #28086 to

metal : single-source fusion table + fusion debug rework (#28164) metal : rework fusion patterns into a single table All fusable op patterns for the Metal backend are now declared once in a fusion tab

cmake : skip PCH for llama-server PCH when using MSVC (#28763) This commit fixes an issue that I introduced when adding PCH (precompiled headers) in Commit 3bcfeb7 ('cmake : add PCH and unity build to

If you can write down how you do your work, you can automate it. Here's what I did to support GitHub's APAC marketing team. The post Marketing ops as code: Automating events from planning to follow-up

model: fix all granite family parameter counts (#28643) model: fix all granite family parameter counts Signed-off-by: Aaron Teo aaron.teo1@ibm.com model: fix additional include, add missing A prefix f

ggml-cpu(s390x): add Q1_0 vector intrinsic support (#28606) ggml-cpu: add ggml_vec_dot_q1_0_q8_0 support Signed-off-by: Aaron Teo aaron.teo1@ibm.com ggml-cpu: clean up variable naming for understandin

ggml-cpu(s390x): add repack support for q4_0 (#28667) ggml-cpu: clean comments Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cp

vulkan : add command-buffer debug labels for GPU profilers (#28101) vulkan : add command-buffer debug labels for GPU profilers Co-authored-by: gabby-zy z2262718160@gmail.com Assisted-by: Claude Code v

memory : avoid allocating V cache for indexer (it's not used) (#28330) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.

vulkan: fall back to shared-memory reduction for dmmv on PowerVR (#28341) The Imagination proprietary Vulkan compiler returns VK_ERROR_UNKNOWN from vkCreateComputePipelines for every dequant mul_mat_v

tests : drop SYCL special-casing in test-backend-ops.cpp (#28688) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46555010 macOS/iOS: macOS Apple Silicon (a

tests : increase tolerance for Add fusion tests (#28691) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46565769 macOS/iOS: macOS Apple Silicon (arm64) mac

models: clean up some dead switch branches in old models (#28669) Some of these if statements were copypastaed in a former refactor and never cleaned up to remove the cases that could never happen any

spec: fix failed to decode mtmd chunk with DFlash (#28587) speculative: fix failed to decode mtmd chunk with DFlash When using DFlash w/ vision models, the drafter memory fails to allocate new tokens

ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (#28687) Move Windows ARM64 CUDA 13.4 builds from the Developer Preview archives to the 13.4.1 GA redistributables Website: https://

vulkan: small M matrix optimizations for qwen (#28457) vulkan: optimize m=1 mul_mat by swapping A/B vulkan: Improve small M perf Allow split_k with small M. Make small vs med tile selection (for coopm

vulkan: use add_alloc_dep to enable topk_moe fusion for prefill (#28422) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46661423 macOS/iOS: macOS Apple Sil

vulkan: use CPU writes in ggml_backend_vk_cpy_tensor_async if the context is idle (#28618) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46709648 macOS/iO

Checking agent-generated code usually means hopping between tabs. Learn how to view diffs, run terminal commands, and preview web apps side by side in the GitHub Copilot app. The post GitHub Copilot a

Add IQ type handling for MoE (#28476) Co-authored-by: cwriter cwriter@localhost Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46124981 macOS/iOS: macOS Ap

tests : use less threads for data initialization (#28325) tests : use 1 thread for data initialization cont : scale threads with number of elements cont : adjust Website: https://llama.app Attestation

vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 (#27471) vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 simplify mmp selectio

vulkan: add dedicated iq4_xs mat-vec shader (#28426) vulkan: add dedicated iq4_xs mat-vec shader Dedicated mul_mat_vec_iq4_xs for the dmmv path, replacing the generic fallback. ~+6-17% token generatio

jinja: treat a null left operand of in as a plain lookup (#28620) Templates that default an optional variable to none and then test its membership in a map hit an error, while the same expression is a

mtmd: propagate video ID to bitmap (#28601) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46201259 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sili

model: fix granite3 moe unknown parameter count (#28632) Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/4620980

args: officially deprecate --mmap|mlock|dio (#28334) Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46228590 ma

CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (#28079) CUDA: add configurable FA quant combinations Assisted-by: Codex remove all flags but , add runtime fal

CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#28552) Recreated from #24546 Co-authored-by: Carl Philipp Klemm carl@uvos.xyz CUDA: pick MMQ tile size against ncols_opt set on t