Search: “llama-cpp-releases”

CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (#27621) CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion

metal : add fa-vec tunings for M1 (#28078) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44172949 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic

ROCm: add radix TOP_K for long rows (#27466) ROCm: add radix TOP_K for long rows Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44178618 macOS/iOS: macOS A

webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_get() implementation (#28045) webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_ge

opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance (#26438) opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse opencl: Q4_K mul_mm 8x8 tile fot In

kv-cache : optimize restoring non-contiguous cells (#27991) kv cache : batch state restore scatter reads per contiguous run When restoring state into non-contiguous destination cells (e.g. a prompt-ca

AVX2: Speed up large batch size prompt processing of IQ models (#27402) Batched gemm for grid IQ quants Style updates and a bit more performance Clean up comments Move code around Vectorize IQ panel d

metal : add concat support for quantized types (#28116) Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/442

CUDA: XOR swizzle flash attn K,V smem fp16 tiles (#25635) CUDA: XOR swizzle flash attn K,V smem fp16 tiles Signed-off-by: ynankani ynankani@nvidia.com Fix use 64bit generic pointer instead of 32bit sh

metal : add fa-vec tunings for M1 Ultra (#28088) metal : add fa-vec tunings for M1 Ultra metal : move M1 Ultra tunings after M1 Max section metal : remove duplicate blank line Website: https://llama.a

metal : add fa-vec tunings for M2 (#27940) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43902029 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic

ggml: allow passing alloc dependencies in graph_optimize (#27301) ggml: allow passing alloc dependencies in graph_optimize add alloc dep tests add TODO about using flat array Website: https://llama.ap

memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.c

metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (#25648) metal : fail closed on mul_mat shapes with missing F16 kernels metal : abort on nil pipeline in encoder_set_pipeline metal : ad

sycl: split long rows in TOP_K instead of one work-group per row (#27847) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43958406 macOS/iOS: macOS Apple Si

hexagon: support for device discovery and create sessions on demand (#27785) hex-devices: add support for lazy session allocation and cleanup dev interfaces Co-authored-by: Marco Colombo mcolombo@qti.

rpc : fix pre-rdma macOS versions (#27815) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43962220 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic

vendor : update cpp-httplib to 0.54.0 (#27919) vendor : update cpp-httplib to 0.54.0 vendor : update cpp-httplib to 0.54.0 and 0.54.1 Website: https://llama.app Attestations: https://github.com/ggml-o

metal : Add fa-vec tuning for M3 Pro (#27963) Related issue: #27668 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43966636 macOS/iOS: macOS Apple Silicon

metal: add fa-vec tunings for M3 Ultra (#27999) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43968762 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple

rpc: fix apple rdma error spew on teardown (#27908) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43970664 macOS/iOS: macOS Apple Silicon (arm64) macOS Ap

ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960) some backends (Metal, SYCL, WebGPU) require additional memory for fleeting data for certain ops, which is reflected in their ge

common: rename --tensor-read-lazy to --lazy-mode, add -lzm shorthand (#27969) Rename the --tensor-read-lazy CLI argument to --lazy-mode, to match the internal lazy_mode parameter, and add a -lzm short

dflash: pass missing NVFP4 scales to attention operations (#28000) DFlash2 NVFP4 draft models produced almost no accepted speculative tokens because the Q, K, V, and output projection scales were not

hip : optimize Q2_0 dot-product path for gfx1201 (#26753) hip/gfx1201: optimize q2_0 vec_dot_q2_0_q8_1 with native amdgcn perm Broadened HIP's Q2_0 perm optimization Remove redundant HIP perm availabi

hip: tune rdna 3 mmq config (#26284) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44012223 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (ar

CUDA: use the fast mm_ids_helper path for any n_expert_used (#27978) The optimized path grouped warp lanes by token and required warp_size % n_expert_used == 0, with a single hardcoded exception paddi

llama: improve TENSOR_READ_LAZY handling (#27837) force lazy tensor on cpu if lazy is on llama: improve TENSOR_READ_LAZY handling Website: https://llama.app Attestations: https://github.com/ggml-org/l

vulkan: Change mul_mat_id to pad K rather than N (#27925) The N padding is needed for mul_mat, but not mul_mat_id. For mul_mat_id, we indirect the row index through a shared memory lookup table which

metal : add fa-vec tunings for M1 Max (#27932) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43859747 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S

vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (#27526) vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs remove one more fastdiv W

sycl: make --fit respect --fit-target better (#27629) improve the --fit algorithm to take into account the actual peak required VRAM for a given context size on a SYCL backend. This includes both prop

metal : add remaining fa-vec tunings for M4 Pro (#27915) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43865923 macOS/iOS: macOS Apple Silicon (arm64) mac

metal : assert shared memory padding (#27951) metal : assert shared memory padding cont : add ref Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43867822 m

opencl: use a better matmul path on two Adreno GPU generations (#27640) opencl: default the Adreno xmem F16xF32 GEMM on for X2E kernel_mul_mm_f16_f32_l4_lm is the slowest matmul this backend has on Ad

model: add DSpark support for Nemotron3.5 (#27804) model: add DSpark support for Nemotron3.5 Update src/models/dflash.cpp Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-

tests : run test-save-load-state across all architectures (#27755) tests : run test-save-load-state across all architectures test-save-load-state previously only ran in ctest against a single download

metal : add fa-vec tunings for M4 Pro (#27824) This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discussions/27668 to add support for additional Apple GP

metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (#27863) metal : add fa-vec tunings for M5 This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discuss

sycl: bind the f16 KV cache in place for the oneDNN SDPA path (#27468) Measured at a live KV length of 34816 (32768 depth plus one 2048 ubatch), on Qwen3.8 27B Q4_K_S: per tensor 4 * 34816 * 256 * 2 B

sycl: use TILE for quantized KV decode on BMG (#26689) Route quantized KV decode to TILE on Xe2 (BMG) only, keep VEC on other archs until validated there. Website: https://llama.app Attestations: http

OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (#27843) OpenVINO Backend: Fuse IM2COL + MatMul convolution into OpenVINO convolution ci:ggml-ov: Skip recurrent state

metal : add fa-vec tunings for M4 (#27875) This adds fa_vec_tuned_table records for Apple M4 to ggml-metal-tuning.cpp. Includes F16, Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0. (M4, 10 GPU Cores) Co-authored-by

Vulkan: add hoisting support for row IDs and expert count in shaders (#26686) vulkan: add hoisting support for row IDs and expert count in shaders use hoisted row ids in coopmat2 vulkan: address revie

ggml : fix conv_transpose_2d for multiple batches (#26132) ggml : fix conv_transpose_2d for multiple batches ggml_compute_forward_conv_transpose_2d_impl only computed the first batch (ne[3] of the des

vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812) vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize is_src_of doesn't treat two views of one tensor as