Search: “models”

graph : avoid reallocations in the 'no cache' path (#30439) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55354972 macOS/iOS: macOS Apple Silicon (arm64)

cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (#30327) ggml-cuda: gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG ggml-cuda: cache env var lookups in static const instead of re-readi

vulkan: allow multiple devices to access gpu-visible host memory (#29741) vulkan: allow multiple devices to access gpu-visible host memory clean up erroneous newlines add Windows support handle nullpt

graph: pad the KQ bias of the no-cache attention (#30446) The no-cache attention pads K, V and the mask to 256, but T5 also adds a relative position bias of [n_tokens, n_tokens], so the add with KQ fa

hexagon: add configurable HTP DCVS/HMX power mode (#30423) hexagon: add configurable HTP DCVS/HMX power mode hexagon: address PR review feedback (drop CLI flag, align QAIRT power modes, comment/log cl

meta: recreate all subgraphs when the arena is reallocated (#30453) meta: recreate all subgraphs when the arena is reallocated chore: revert added test cases Website: https://llama.app Attestations: h

vulkan: Optimize pool_2d when the pooling window is large (#30119) When the window is larger than a subgroup, use a whole subgroup to compute each output. Website: https://llama.app Attestations: http

kleidiai: add SME2 Q4_K and Q6_K matmul support (#29219) Change-Id: Ib4aa4c9814c010f9cf084783cc80612e364f9a27 Change-Id: I406a6606c3fea51f8cd32331c1ed71466845d363 Change-Id: Ib4aa4c9814c010f9cf084783c

ggml : simplify backend search path error handling (#28367) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestatio

hexagon: add IM2COL_3D (#30305) hexagon: add IM2COL_3D support Assisted-by: OpenCode hexagon: add DMA paths for IM2COL_3D Assisted-by: OpenCode hexagon: avoid scalar fp16 conversions in IM2COL Assiste

vulkan: fix buffer logging on 32-bit platforms after source split (#30326) Move the vk::Buffer stream operator to the shared header so it is visible to ggml-vulkan-debug.cpp. Website: https://llama.ap

ggml: Change default visibility of symbols to hidden (#30014) See discussion in #29997. This prevents a handful of internal symbols leaking via GGML_API. Website: https://llama.app Attestations: https

vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size (#30164) vulkan: specialize quant block size for CM2 matmul vulkan: use ternary expressions to avoid switch lowering f

sycl: guard DMMV reads at row tails (#29338) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54988744 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sil

llama : report compute graph reallocations during inference (#30329) Add an optional ggml_backend_sched_realloc_callback that user code registers with ggml_backend_sched_set_realloc_callback to be not

metal : fix mul_mat_id with duplicate ids (#30373) metal : fix mul_mat_id limits and kernel name metal : fix mul_mat_id with duplicate ids kernel_mul_mm_id_map0 assumed at most one match per token and

tests : link each test against only the libraries it uses (#30299) Give llama_build() and llama_build_and_test() a LINK keyword (default: llama-common) so call sites can declare what a test actually n

vulkan: pack FMA for f16 accumulation, no dot product op (#29877) vulkan: pack FMA for f16 accumulation, no dot product op vulkan: dot_product packs two rows per f16 FMA vulkan: rename dot_product to

metal: fix metal tensor kernel_mul_mm error for large tensor (#28748) metal: fix metal tensor kernel_mul_mm error for large tensor Assisted-by: Codex Update ggml/src/ggml-metal/kernels/mul_mm.metal Co

server: add component tests (#30312) server: add component tests nits add codeowners Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54782864 macOS/iOS: mac

vulkan: handle mul_mat_id duplicates in the prepass rather than looping (#29998) vulkan: compute every row of an expert in mul_mm_id when ids repeat vulkan: handle mul_mat_id duplicates in the prepass

cpu: fix AVX2 CPUs not using F16C when available (#30270) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54796975 macOS/iOS: macOS Apple Silicon (arm64) ma

rpc : turn GGML_RPC_DEBUG into a verbosity level and add logs GGML_RPC_DEBUG is now parsed as a number: 0/unset disables debug logs, 1-3 emit increasingly detailed output (events, per-command trace, t

opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rms_norm (#30310) opencl: fallback when Q4_K weight images exceed device limits Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com

opencl: improve fa, allow dk512 for gemma-4, improve dk64 (#30266) opencl: enable Gemma-4 E4B GPU decode Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com opencl: extend Gemma-4 GPU decode to E2B

sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering (#29809) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54540406 macOS/iOS: macO

opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin (#30187) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5457

cmake(s390x): disable z17 target for unsupported compilers (#30297) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54655455 macOS/iOS: macOS Apple Silicon

server: leave a busy slot untouched when a request pins it (#30295) A request asking for a busy id_slot still ran the prompt cache update on that slot before being deferred. When the RAM cache held a

hexagon: fix IM2COL patch-embed DMA ring overflow (#30189) hexagon: fix IM2COL patch-embed DMA ring overflow The exact-tiling (stride == kernel, no pad/dilation) IM2COL DMA kernel issues ICKH DDR->VTC

CUDA: pass src1 precision to host MMQ config helpers (#30168) CUDA: pass src1 precision to host MMQ config helpers Signed-off-by: ynankani ynankani@nvidia.com make src1 prec explicit Signed-off-by: yn

metal: add the 128/96 flash attention kernels, accept every compiled pair (#30209) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54260937 macOS/iOS: macOS

server: use port 9931 by default (#30159) server: use port 9931 by default revert unrelated changes Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54269123

musa: drop mp_21 from the default architectures (#30203) musa: drop mp_21 from the default architectures Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Address review comments Signed-off-by: Xiao

vulkan: fix rms_norm workgroup count overflow (#30145) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54337638 macOS/iOS: macOS Apple Silicon (arm64) macOS

webgpu: use 2D workgroup dispatch for all the ops which use 1D dispatch (e.g., rms_norm) (#30219) remove 1D workgroups dispatching formatting Website: https://llama.app Attestations: https://github.co

meta : handle host views (#30217) meta : handle views of tensors allocated on the host A view shares the memory of its view_src, so ggml-alloc never allocates a view in the buffer of the split it land

ggml: fix OOB write in ggml_acc with negative offset (#30135) ggml_acc_impl narrowed a size_t offset to int32_t without checking that it fits, so a large offset could truncate to a negative int32_t. T

llama: keep the backend sampling graph static across ubatches (#30223) The reserve builds n_outputs_max_per_seq sampling chains per sampler, while a decode built one per output row, so the graph chang

chat : refactor API (#30210) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54392536 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, Kle

opencl: fix kernel compilation for a6x GPUs (#30176) opencl: skip kernel_cpy_f32_f32_pack on A6X to avoid shader compiler crash The A6x compiler backend found in iot device with a623 (E031.50.31.01) c

CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807) CUDA: fuse copy of updated state snapshots into recurrent cache with ssm_scan CUDA: remove redundant cuda copies with K==1 (non spec-dec) sce

llama-bench: respect -fitc if bigger than required benchmark size (#28331) Assisted-By: opencode,llama.cpp,Qwen3.6-35B-A3B,Qwen3.8-27B Website: https://llama.app Attestations: https://github.com/ggml-

graph : reorder get_rows for embeddings (#30160) graph : reorder get_rows for embeddings cont : fix gemma4 and improve input embedding construction logic cont : add TODO for lora cont : fix raw embedd

CUDA: fix round issue, under MSVC the CPU and GPU agree (#30229) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54453988 macOS/iOS: macOS Apple Silicon (ar

vendor: apply deep nested json patch from upstream (#30253) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54488539 macOS/iOS: macOS Apple Silicon (arm64)