b11620
opencl: fix q8_0 dp4a mul_mat_id and mark PREC_F32 mul_mat_id unsupported for adreno (#30365) opencl: fix min handling for moe q8_0 dp4a GEMM Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com open
Knowledge catalogue
opencl: fix q8_0 dp4a mul_mat_id and mark PREC_F32 mul_mat_id unsupported for adreno (#30365) opencl: fix min handling for moe q8_0 dp4a GEMM Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com open
opencl: use buf+offset instead of subbuffer for adreno moe expert sort (#30418) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55971785 macOS/iOS: macOS Ap
jinja : write floats in tojson as Python does (#30362) tojson printed floats through a C++ stream (6 significant digits, no '.0'), so 48.858844 became 48.8588 and 1.0 became 1 in tool definitions, too
cuda : use SIMT implicit GEMM conv2d on AMD without WMMA (#30325) cuda : use SIMT implicit GEMM conv2d on AMD without WMMA Assisted-by: Claude Opus cuda : update conv2d SIMT comment cuda : skip SIMT c
ggml : improve RPC tensor transfer logs (#30481) add rpc_log_format_size() and print byte counts in KiB/MiB/GiB across the RPC client, server and transport traces add rpc_log_format_tensor() and inclu
arXiv:2610.15864v1 Announce Type: cross Abstract: Large-scale personalization is shifting from embedding-table retrieval to LLM-based recommenders that generate the Semantic ID (SID) of the item to re
arXiv:2601.07674v3 Announce Type: replace-cross Abstract: Random walk (RW)-based algorithms have long been popular in distributed systems due to low overheads and scalability, with recent growing appl
hexagon: extend scalar DMA broadcasting support for binary ops (#30245) tests: cover weighted scalar broadcasts Assisted-by: OpenCode ggml : support extended scalar DMA binary ops Assisted-by: OpenCod
models : remove redundant ggml_cont calls now supported by all unary kernels (#30198) models: remove redundant ggml_cont calls now supported by CUDA unary kernels models: restore cont_2d after sigmoid
ggml-zdnn : introduce Q8_0 mat mul (#29836) ggml-zdnn: initial prototype of q8_0 matmul Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: update zdnn docs Signed-off-by: Aaron Teo aaron.teo1@ibm.com S
cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (#30327) ggml-cuda: gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG ggml-cuda: cache env var lookups in static const instead of re-readi
vulkan: allow multiple devices to access gpu-visible host memory (#29741) vulkan: allow multiple devices to access gpu-visible host memory clean up erroneous newlines add Windows support handle nullpt
tests : consolidate context state tests into test-llama-context (#30023) tests : consolidate the context state tests into test-llama-context test-save-load-state, test-recurrent-state-rollback and tes
graph: skip the LM head for pooled embeddings (#30445) graph: skip the LM head for pooled embeddings A context that extracts pooled embeddings outputs one vector per sequence, but every model still ex
model: load YaRN beta metadata for laguna models (#30331) Assisted-by: Codex Co-authored-by: Adam Caldwell accaldwell@users.noreply.github.com Website: https://llama.app Attestations: https://github.c
graph: pad the KQ bias of the no-cache attention (#30446) The no-cache attention pads K, V and the mask to 256, but T5 also adds a relative position bias of [n_tokens, n_tokens], so the add with KQ fa
hexagon: add configurable HTP DCVS/HMX power mode (#30423) hexagon: add configurable HTP DCVS/HMX power mode hexagon: address PR review feedback (drop CLI flag, align QAIRT power modes, comment/log cl
meta: recreate all subgraphs when the arena is reallocated (#30453) meta: recreate all subgraphs when the arena is reallocated chore: revert added test cases Website: https://llama.app Attestations: h
vulkan: Optimize pool_2d when the pooling window is large (#30119) When the window is larger than a subgroup, use a whole subgroup to compute each output. Website: https://llama.app Attestations: http
kleidiai: add SME2 Q4_K and Q6_K matmul support (#29219) Change-Id: Ib4aa4c9814c010f9cf084783cc80612e364f9a27 Change-Id: I406a6606c3fea51f8cd32331c1ed71466845d363 Change-Id: Ib4aa4c9814c010f9cf084783c
ggml : simplify backend search path error handling (#28367) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestatio
hexagon: add IM2COL_3D (#30305) hexagon: add IM2COL_3D support Assisted-by: OpenCode hexagon: add DMA paths for IM2COL_3D Assisted-by: OpenCode hexagon: avoid scalar fp16 conversions in IM2COL Assiste
vulkan: fix buffer logging on 32-bit platforms after source split (#30326) Move the vk::Buffer stream operator to the shared header so it is visible to ggml-vulkan-debug.cpp. Website: https://llama.ap
ggml: Change default visibility of symbols to hidden (#30014) See discussion in #29997. This prevents a handful of internal symbols leaking via GGML_API. Website: https://llama.app Attestations: https
vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size (#30164) vulkan: specialize quant block size for CM2 matmul vulkan: use ternary expressions to avoid switch lowering f
llama : report compute graph reallocations during inference (#30329) Add an optional ggml_backend_sched_realloc_callback that user code registers with ggml_backend_sched_set_realloc_callback to be not
metal : fix mul_mat_id with duplicate ids (#30373) metal : fix mul_mat_id limits and kernel name metal : fix mul_mat_id with duplicate ids kernel_mul_mm_id_map0 assumed at most one match per token and
tests : link each test against only the libraries it uses (#30299) Give llama_build() and llama_build_and_test() a LINK keyword (default: llama-common) so call sites can declare what a test actually n
model: support vllm decision 2 model (#30378) models: support pplx-decider vllm d2 update clarify naming rule stricter validation fix ci rm redundant filter_tensors Website: https://llama.app Attestat
model: support vllm d3 models (#30383) models: support pplx-decider vllm d2 update clarify naming rule stricter validation d3 Website: https://llama.app Attestations: https://github.com/ggml-org/llama
vulkan: pack FMA for f16 accumulation, no dot product op (#29877) vulkan: pack FMA for f16 accumulation, no dot product op vulkan: dot_product packs two rows per f16 FMA vulkan: rename dot_product to
metal: fix metal tensor kernel_mul_mm error for large tensor (#28748) metal: fix metal tensor kernel_mul_mm error for large tensor Assisted-by: Codex Update ggml/src/ggml-metal/kernels/mul_mm.metal Co
model: fix vllm d3 multi seq case (#30408) model: fix vllm d3 multi seq case add TODO Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55219202 macOS/iOS: ma
I've made a few posts recently, just talking about the hardware I've got and the model I use. Pair of P100s, Unsloth Qwen3.6 35b A3B UD Q4 K XL. There's been a ton of talk of model and hardware specif
server: add component tests (#30312) server: add component tests nits add codeowners Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54782864 macOS/iOS: mac
vulkan: handle mul_mat_id duplicates in the prepass rather than looping (#29998) vulkan: compute every row of an expert in mul_mm_id when ids repeat vulkan: handle mul_mat_id duplicates in the prepass
model : support for Prism Bonsai 2 27B (#29600) Runtime support for Prism Bonsai 2 27B Assisted-by: Claude Code address prism hadamard runtime feedback move hadamard tensors into method, define folded
opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rms_norm (#30310) opencl: fallback when Q4_K weight images exceed device limits Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com
opencl: improve fa, allow dk512 for gemma-4, improve dk64 (#30266) opencl: enable Gemma-4 E4B GPU decode Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com opencl: extend Gemma-4 GPU decode to E2B
Currently contemplating adding either M5 Ultra 256GB or dual DGX Spark in addition to existing 5090. I have been researching for past few days and have zeroed in on dual sparks because higher throughp
sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering (#29809) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54540406 macOS/iOS: macO
opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin (#30187) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5457
model : support MiniCPM-V 4.7 (#29416) mtmd : add MiniCPM-V 4.7 support Signed-off-by: tc-mb tianchi_cai@icloud.com model : allow mrope time from an extra position slot Signed-off-by: tc-mb tianchi_ca
server: leave a busy slot untouched when a request pins it (#30295) A request asking for a busy id_slot still ran the prompt cache update on that slot before being deferred. When the RAM cache held a
Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted
hexagon: fix IM2COL patch-embed DMA ring overflow (#30189) hexagon: fix IM2COL patch-embed DMA ring overflow The exact-tiling (stride == kernel, no pad/dilation) IM2COL DMA kernel issues ICKH DDR->VTC
CUDA: pass src1 precision to host MMQ config helpers (#30168) CUDA: pass src1 precision to host MMQ config helpers Signed-off-by: ynankani ynankani@nvidia.com make src1 prec explicit Signed-off-by: yn
metal: add the 128/96 flash attention kernels, accept every compiled pair (#30209) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54260937 macOS/iOS: macOS
server: use port 9931 by default (#30159) server: use port 9931 by default revert unrelated changes Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54269123
ci: fix Models Backend webgpu by supporting GGML_OP_DUP (#30216) The mixed batch path of PR-29622 writes the token rows with set_rows into a dup of the embeddings. WebGPU did not support DUP, so the d
musa: drop mp_21 from the default architectures (#30203) musa: drop mp_21 from the default architectures Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Address review comments Signed-off-by: Xiao
webgpu: use 2D workgroup dispatch for all the ops which use 1D dispatch (e.g., rms_norm) (#30219) remove 1D workgroups dispatching formatting Website: https://llama.app Attestations: https://github.co
meta : handle host views (#30217) meta : handle views of tensors allocated on the host A view shares the memory of its view_src, so ggml-alloc never allocates a view in the buffer of the split it land
ggml: fix OOB write in ggml_acc with negative offset (#30135) ggml_acc_impl narrowed a size_t offset to int32_t without checking that it fits, so a large offset could truncate to a negative int32_t. T
llama: keep the backend sampling graph static across ubatches (#30223) The reserve builds n_outputs_max_per_seq sampling chains per sampler, while a decode built one per output row, so the graph chang
model : use exact GELU for ModernBERT encoders (#30108) model : use exact GELU for ModernBERT encoders Assisted-by: Codex model : keep tanh GELU aliases on ggml_geglu Assisted-by: Claude Opus 5.5 mode
opencl: fix kernel compilation for a6x GPUs (#30176) opencl: skip kernel_cpy_f32_f32_pack on A6X to avoid shader compiler crash The A6x compiler backend found in iot device with a623 (E031.50.31.01) c
CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807) CUDA: fuse copy of updated state snapshots into recurrent cache with ssm_scan CUDA: remove redundant cuda copies with K==1 (non spec-dec) sce
llama-bench: respect -fitc if bigger than required benchmark size (#28331) Assisted-By: opencode,llama.cpp,Qwen3.6-35B-A3B,Qwen3.8-27B Website: https://llama.app Attestations: https://github.com/ggml-
graph : reorder get_rows for embeddings (#30160) graph : reorder get_rows for embeddings cont : fix gemma4 and improve input embedding construction logic cont : add TODO for lora cont : fix raw embedd