b10946
ggml-cpu(s390x): guard VXE-only repack helpers (#28775) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47190488 macOS/iOS: macOS Apple Silicon (arm64) macO
Knowledge catalogue
ggml-cpu(s390x): guard VXE-only repack helpers (#28775) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47190488 macOS/iOS: macOS Apple Silicon (arm64) macO
models : guard the expert FFN size fallback in nemotron-h against a zero divisor (#28779) The NextN/MTP tail loop derives the expert FFN size as n_ff/n_expert_used when expert_feed_forward_length give
tests : exclude HY_V4 from WebGPU test-llama-archs tests (#28855) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/a
ggml-webgpu: Update to a recent version of Dawn (#28683) ggml-webgpu: Update to a recent version of Dawn No module scanning Accept review suggestion to update comment Co-authored-by: Masashi Yoshimura
hexagon: support for multi-device model split (aka row-split) (#28589) hex-row-split: add support for multi-device row spliting Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com hex-mdev: add work
webgpu: align tensor bindings to the type block size (#28382) Walk the binding offset back until the distance to the tensor is a whole number of blocks, so block quantized views get a valid element of
opencl: add bin kernel kernel_gemm_noshuffle_q4_k_f32_32b_trans_ila_a8_bin (#28677) opencl: add A8 Q4_K non-MoE binary kernel opencl: fix layout compatibility opencl: rename binary kernel selection he
opencl: fix several bugs where the backend aborts (#27630) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47037447 macOS/iOS: macOS Apple Silicon (arm64) m
server: frame the router child state command as a whole line (#28747) The child writes its state commands on stdout while the logger writes on stderr, and both share a single pipe. The logger emits th
syscl : Handle (fail gracefully) unsupported tq1_0 quants (#28681) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47043623 macOS/iOS: macOS Apple Silicon (
vendor : update cpp-httplib to 0.56.0 (#28787) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47045738 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
ggml-cuda: hip add specific config table for AMD GCN (#27841) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47052867 macOS/iOS: macOS Apple Silicon (arm64
server : allow model downloads at model limit fix issue #26809 (#28530) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47055500 macOS/iOS: macOS Apple Sili
ui : add cache (#28802) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47065031 macOS/iOS: macOS Apple S
cmake: leave the timestamp out of precompiled headers on clang (#28816) Clang stores the modification time of the precompiled header sources inside the header and refuses the header when they differ.
jinja : support dot property integer literals (#28817) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47107718 macOS/iOS: macOS Apple Silicon (arm64) macOS
opencl: add A8 Q4_0 mm binary kernel support (#28268) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46774832 macOS/iOS: macOS Apple Silicon (arm64) macOS
vulkan: fix data race and OOB access in argsort(large) (#28705) argsort had a data race in the inner loop, which VVL caught. But I don't think this was causing failures in practice. argsort_large has
CUDA/HIP: Flash Attention tuning (gfx1201) (#28102) HIP: enable mma FA for head size 256 on RDNA4, tune configs Assisted-by: Claude Assisted-by: Codex HIP: prefer whole-tile FA grids over stream-k on
server: fix speculation after an image (#28715) server: fix speculation after an image Pass the actual position to the drafter after an image, instead of the token count. Affects every drafter, not ju
model : fix MTP context kv cache allocation for deepseek2, glm4moe, … (#28630) model : fix MTP context kv cache allocation for deepseek2, glm4moe, cohere2moe architectures (#28626) model: add inverse
metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 Generalize the row split from #28086 to
metal : single-source fusion table + fusion debug rework (#28164) metal : rework fusion patterns into a single table All fusable op patterns for the Metal backend are now declared once in a fusion tab
cmake : skip PCH for llama-server PCH when using MSVC (#28763) This commit fixes an issue that I introduced when adding PCH (precompiled headers) in Commit 3bcfeb7 ('cmake : add PCH and unity build to
If you can write down how you do your work, you can automate it. Here's what I did to support GitHub's APAC marketing team. The post Marketing ops as code: Automating events from planning to follow-up
model: fix all granite family parameter counts (#28643) model: fix all granite family parameter counts Signed-off-by: Aaron Teo aaron.teo1@ibm.com model: fix additional include, add missing A prefix f
ggml-cpu(s390x): add Q1_0 vector intrinsic support (#28606) ggml-cpu: add ggml_vec_dot_q1_0_q8_0 support Signed-off-by: Aaron Teo aaron.teo1@ibm.com ggml-cpu: clean up variable naming for understandin
ggml-cpu(s390x): add repack support for q4_0 (#28667) ggml-cpu: clean comments Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cp
vulkan : add command-buffer debug labels for GPU profilers (#28101) vulkan : add command-buffer debug labels for GPU profilers Co-authored-by: gabby-zy z2262718160@gmail.com Assisted-by: Claude Code v
memory : avoid allocating V cache for indexer (it's not used) (#28330) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.
vulkan: fall back to shared-memory reduction for dmmv on PowerVR (#28341) The Imagination proprietary Vulkan compiler returns VK_ERROR_UNKNOWN from vkCreateComputePipelines for every dequant mul_mat_v
tests : drop SYCL special-casing in test-backend-ops.cpp (#28688) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46555010 macOS/iOS: macOS Apple Silicon (a
tests : increase tolerance for Add fusion tests (#28691) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46565769 macOS/iOS: macOS Apple Silicon (arm64) mac
models: clean up some dead switch branches in old models (#28669) Some of these if statements were copypastaed in a former refactor and never cleaned up to remove the cases that could never happen any
spec: fix failed to decode mtmd chunk with DFlash (#28587) speculative: fix failed to decode mtmd chunk with DFlash When using DFlash w/ vision models, the drafter memory fails to allocate new tokens
ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (#28687) Move Windows ARM64 CUDA 13.4 builds from the Developer Preview archives to the 13.4.1 GA redistributables Website: https://
vulkan: small M matrix optimizations for qwen (#28457) vulkan: optimize m=1 mul_mat by swapping A/B vulkan: Improve small M perf Allow split_k with small M. Make small vs med tile selection (for coopm
vulkan: use add_alloc_dep to enable topk_moe fusion for prefill (#28422) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46661423 macOS/iOS: macOS Apple Sil
vulkan: use CPU writes in ggml_backend_vk_cpy_tensor_async if the context is idle (#28618) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46709648 macOS/iO
Checking agent-generated code usually means hopping between tabs. Learn how to view diffs, run terminal commands, and preview web apps side by side in the GitHub Copilot app. The post GitHub Copilot a
Add IQ type handling for MoE (#28476) Co-authored-by: cwriter cwriter@localhost Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46124981 macOS/iOS: macOS Ap
tests : use less threads for data initialization (#28325) tests : use 1 thread for data initialization cont : scale threads with number of elements cont : adjust Website: https://llama.app Attestation
vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 (#27471) vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 simplify mmp selectio
vulkan: add dedicated iq4_xs mat-vec shader (#28426) vulkan: add dedicated iq4_xs mat-vec shader Dedicated mul_mat_vec_iq4_xs for the dmmv path, replacing the generic fallback. ~+6-17% token generatio
jinja: treat a null left operand of in as a plain lookup (#28620) Templates that default an optional variable to none and then test its membership in a map hit an error, while the same expression is a
mtmd: propagate video ID to bitmap (#28601) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46201259 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sili
model: fix granite3 moe unknown parameter count (#28632) Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/4620980
args: officially deprecate --mmap|mlock|dio (#28334) Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/46228590 ma
CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (#28079) CUDA: add configurable FA quant combinations Assisted-by: Codex remove all flags but , add runtime fal
CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#28552) Recreated from #24546 Co-authored-by: Carl Philipp Klemm carl@uvos.xyz CUDA: pick MMQ tile size against ncols_opt set on t
llama : use int32_t for llama_sampler_chain_n return type (#28631) Contributes to #4574 Co-authored-by: linsen linsen@insta360.com Website: https://llama.app Attestations: https://github.com/ggml-org/
vulkan: Convert FILL to distribute workgroups in 2D to avoid exceeding maxComputeWorkGroupCount (#28592) divide workload to 2D This is to workaround FILL exceeding maxComputeWorkGroupCount for Intel G
vulkan: use spec constant for matrix matrix multiplication A-type (#25773) vulkan: use spec constant for mul mat type_a vulkan: use map for mul_mm shapes cleanup fix indentation fix cm2 and shmem init
Use Ollama models in ChatGPT Desktop Ollama models can now be used directly in ChatGPT Desktop, so you can keep your existing workflow while running open models. Setup is available from the Ollama app
hexagon: add RELU and LEAKY_RELU ops (#28585) hexagon: add RELU op hexagon: add LEAKY_RELU op too Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45844223 m
model : support Kimi-K3 recurrent-state rollback (#28466) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45866131 macOS/iOS: macOS Apple Silicon (arm64) ma
opencl: properly handle non-contiguous inputs to conv2d (#28503) opencl: fix conv2d non-contiguous strides opencl: format Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp
chat : split specialized parsers into common/parsers (#27764) chat : split specialized parsers into common/parsers Move the 14 dedicated template parsers out of chat.cpp into one file each under commo
Fix Vulkan-Hpp handle usage on 32-bit targets. (#22892) On 32-bit platforms, Vulkan non-dispatchable handles such as VkBuffer are represented as uint64_t, and Vulkan-Hpp disables implicit conversions
vulkan : fuse UNARY(GELU|SIGMOID|SILU|SOFTPLUS) + MUL (#27220) vulkan : fuse UNARY(SIGMOID|SILU|SOFTPLUS) + MUL vulkan : fuse UNARY(SIGMOID|SILU|SOFTPLUS) + MUL implement fusion in unary.comp behind U