Search: “apple”

cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (#30327) ggml-cuda: gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG ggml-cuda: cache env var lookups in static const instead of re-readi

vulkan: allow multiple devices to access gpu-visible host memory (#29741) vulkan: allow multiple devices to access gpu-visible host memory clean up erroneous newlines add Windows support handle nullpt

tests : consolidate context state tests into test-llama-context (#30023) tests : consolidate the context state tests into test-llama-context test-save-load-state, test-recurrent-state-rollback and tes

graph: skip the LM head for pooled embeddings (#30445) graph: skip the LM head for pooled embeddings A context that extracts pooled embeddings outputs one vector per sequence, but every model still ex

model: load YaRN beta metadata for laguna models (#30331) Assisted-by: Codex Co-authored-by: Adam Caldwell accaldwell@users.noreply.github.com Website: https://llama.app Attestations: https://github.c

graph: pad the KQ bias of the no-cache attention (#30446) The no-cache attention pads K, V and the mask to 256, but T5 also adds a relative position bias of [n_tokens, n_tokens], so the add with KQ fa

hexagon: add configurable HTP DCVS/HMX power mode (#30423) hexagon: add configurable HTP DCVS/HMX power mode hexagon: address PR review feedback (drop CLI flag, align QAIRT power modes, comment/log cl

meta: recreate all subgraphs when the arena is reallocated (#30453) meta: recreate all subgraphs when the arena is reallocated chore: revert added test cases Website: https://llama.app Attestations: h

vulkan: Optimize pool_2d when the pooling window is large (#30119) When the window is larger than a subgroup, use a whole subgroup to compute each output. Website: https://llama.app Attestations: http

kleidiai: add SME2 Q4_K and Q6_K matmul support (#29219) Change-Id: Ib4aa4c9814c010f9cf084783cc80612e364f9a27 Change-Id: I406a6606c3fea51f8cd32331c1ed71466845d363 Change-Id: Ib4aa4c9814c010f9cf084783c

ggml : simplify backend search path error handling (#28367) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestatio

hexagon: add IM2COL_3D (#30305) hexagon: add IM2COL_3D support Assisted-by: OpenCode hexagon: add DMA paths for IM2COL_3D Assisted-by: OpenCode hexagon: avoid scalar fp16 conversions in IM2COL Assiste

vulkan: fix buffer logging on 32-bit platforms after source split (#30326) Move the vk::Buffer stream operator to the shared header so it is visible to ggml-vulkan-debug.cpp. Website: https://llama.ap

ggml: Change default visibility of symbols to hidden (#30014) See discussion in #29997. This prevents a handful of internal symbols leaking via GGML_API. Website: https://llama.app Attestations: https

vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size (#30164) vulkan: specialize quant block size for CM2 matmul vulkan: use ternary expressions to avoid switch lowering f

llama : report compute graph reallocations during inference (#30329) Add an optional ggml_backend_sched_realloc_callback that user code registers with ggml_backend_sched_set_realloc_callback to be not

metal : fix mul_mat_id with duplicate ids (#30373) metal : fix mul_mat_id limits and kernel name metal : fix mul_mat_id with duplicate ids kernel_mul_mm_id_map0 assumed at most one match per token and

tests : link each test against only the libraries it uses (#30299) Give llama_build() and llama_build_and_test() a LINK keyword (default: llama-common) so call sites can declare what a test actually n

model: support vllm decision 2 model (#30378) models: support pplx-decider vllm d2 update clarify naming rule stricter validation fix ci rm redundant filter_tensors Website: https://llama.app Attestat

model: support vllm d3 models (#30383) models: support pplx-decider vllm d2 update clarify naming rule stricter validation d3 Website: https://llama.app Attestations: https://github.com/ggml-org/llama

vulkan: pack FMA for f16 accumulation, no dot product op (#29877) vulkan: pack FMA for f16 accumulation, no dot product op vulkan: dot_product packs two rows per f16 FMA vulkan: rename dot_product to

metal: fix metal tensor kernel_mul_mm error for large tensor (#28748) metal: fix metal tensor kernel_mul_mm error for large tensor Assisted-by: Codex Update ggml/src/ggml-metal/kernels/mul_mm.metal Co

model: fix vllm d3 multi seq case (#30408) model: fix vllm d3 multi seq case add TODO Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55219202 macOS/iOS: ma

I've made a few posts recently, just talking about the hardware I've got and the model I use. Pair of P100s, Unsloth Qwen3.6 35b A3B UD Q4 K XL. There's been a ton of talk of model and hardware specif

server: add component tests (#30312) server: add component tests nits add codeowners Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54782864 macOS/iOS: mac

vulkan: handle mul_mat_id duplicates in the prepass rather than looping (#29998) vulkan: compute every row of an expert in mul_mm_id when ids repeat vulkan: handle mul_mat_id duplicates in the prepass

model : support for Prism Bonsai 2 27B (#29600) Runtime support for Prism Bonsai 2 27B Assisted-by: Claude Code address prism hadamard runtime feedback move hadamard tensors into method, define folded

opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rms_norm (#30310) opencl: fallback when Q4_K weight images exceed device limits Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com

opencl: improve fa, allow dk512 for gemma-4, improve dk64 (#30266) opencl: enable Gemma-4 E4B GPU decode Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com opencl: extend Gemma-4 GPU decode to E2B

Currently contemplating adding either M5 Ultra 256GB or dual DGX Spark in addition to existing 5090. I have been researching for past few days and have zeroed in on dual sparks because higher throughp

sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reordering (#29809) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54540406 macOS/iOS: macO

opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_gemm_moe_q6_k_q8_1_dp4a_bin (#30187) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5457

model : support MiniCPM-V 4.7 (#29416) mtmd : add MiniCPM-V 4.7 support Signed-off-by: tc-mb tianchi_cai@icloud.com model : allow mrope time from an extra position slot Signed-off-by: tc-mb tianchi_ca

server: leave a busy slot untouched when a request pins it (#30295) A request asking for a busy id_slot still ran the prompt cache update on that slot before being deferred. When the RAM cache held a

Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted

hexagon: fix IM2COL patch-embed DMA ring overflow (#30189) hexagon: fix IM2COL patch-embed DMA ring overflow The exact-tiling (stride == kernel, no pad/dilation) IM2COL DMA kernel issues ICKH DDR->VTC

CUDA: pass src1 precision to host MMQ config helpers (#30168) CUDA: pass src1 precision to host MMQ config helpers Signed-off-by: ynankani ynankani@nvidia.com make src1 prec explicit Signed-off-by: yn

metal: add the 128/96 flash attention kernels, accept every compiled pair (#30209) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54260937 macOS/iOS: macOS

server: use port 9931 by default (#30159) server: use port 9931 by default revert unrelated changes Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54269123

ci: fix Models Backend webgpu by supporting GGML_OP_DUP (#30216) The mixed batch path of PR-29622 writes the token rows with set_rows into a dup of the embeddings. WebGPU did not support DUP, so the d

musa: drop mp_21 from the default architectures (#30203) musa: drop mp_21 from the default architectures Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Address review comments Signed-off-by: Xiao

webgpu: use 2D workgroup dispatch for all the ops which use 1D dispatch (e.g., rms_norm) (#30219) remove 1D workgroups dispatching formatting Website: https://llama.app Attestations: https://github.co

meta : handle host views (#30217) meta : handle views of tensors allocated on the host A view shares the memory of its view_src, so ggml-alloc never allocates a view in the buffer of the split it land

ggml: fix OOB write in ggml_acc with negative offset (#30135) ggml_acc_impl narrowed a size_t offset to int32_t without checking that it fits, so a large offset could truncate to a negative int32_t. T

llama: keep the backend sampling graph static across ubatches (#30223) The reserve builds n_outputs_max_per_seq sampling chains per sampler, while a decode built one per output row, so the graph chang

model : use exact GELU for ModernBERT encoders (#30108) model : use exact GELU for ModernBERT encoders Assisted-by: Codex model : keep tanh GELU aliases on ggml_geglu Assisted-by: Claude Opus 5.5 mode