b11257
common : use fs::path for config dir (#29649) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51176378 ma
Knowledge catalogue
common : use fs::path for config dir (#29649) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51176378 ma
common : stop accepting draft tokens at EOG (#29638) common : stop accepting draft tokens at EOG cont : remove the test Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/a
hexagon: add FP32 GELU_ERF and GEGLU_ERF support (#29631) hexagon: add FP32 GELU_ERF and GEGLU_ERF support hex-erf: reduce register pressure in kernels Website: https://llama.app Attestations: https:/
ggml : require input tensors to be GGML_OP_NONE (#29647) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51266318 macOS/iOS: macOS Apple Silicon (arm64) mac
ggml : accumulate f16 dot products in f32 on AVX512-FP16 (#29545) Supersedes #29530 Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-
vocab : keep NORMAL in PLaMo-2 and PLaMo-3 (#29580) vocab : keep NORMAL in PLaMo-2 and PLaMo-3 The PLaMo-2 and PLaMo-3 vocabularies mark as NORMAL. Current EOG token heuristic matched it by text and a
ggml : fix c++ odr by properly using GGML_COMMON_DECL_CPP (#29504) fix c++ odr by properly using GGML_COMMON_DECL_CPP using actual field rather than macro Co-authored-by: Georgi Gerganov ggerganov@gma
vulkan: MOE aware mat_mul_id tile selection (#29182) mut_mul_id selected its matmul tile with total token count. For MoE dispatch grid the true N per workgroup is per-expert rows. At pp128 on Sarvam 3
vulkan : Load F32 A matrix 2 at a time when its 2-aligned (#29254) It turns out Intel doesn't particularly like loading F32s one at a time and we already have the _2aliagned load logic in mul_mat_vec,
vulkan: fix wrong results when a mul_mat reads a slice of a larger cache (#28956) vulkan: read the batch stride of an in place src0 from nb[2] A dim01 contiguous tensor can still be a view whose batch
tests : fix ggml init (#29554) tests : init ggml for test-recurrent-state-rollback cont : same for test-save-load-state cont : add to test-state-restore-fragmented + add TODOs Website: https://llama.a
Enables Windows ARM64 build with MSVC cl.exe (#28362) can reproduce the issue vlad sees fix fma issue drop volatile fix volatile runtime task add arm flag if needed fix hsum compile error fix syntax i
context : do not re-reserve the scheduler when toggling causal_attn (#28751) context : do not re-reserve the scheduler when toggling causal_attn llama_context::set_causal_attn() marks the scheduler to
metal: support left and circular padding in GGML_OP_PAD (#29561) metal: support left and circular padding in GGML_OP_PAD Align Metal with CPU, CUDA and Vulkan: shift the source coordinates by the left
HIP: fix template skip for DKQ > 256 mfma kernels (#29559) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50767985 macOS/iOS: macOS Apple Silicon (arm64) m
ggml-cpu: enable tiled flash attention for non-vector-multiple head dims on x86 (#29423) ggml-cpu: enable tiled flash attention for non-vector-multiple head dims on x86 add AVX2 support for masked loa
tests : refactor test-recurrent-state-rollback (#29426) tests : use llama_context_ptr in test-recurrent-state-rollback Replace raw llama_context pointers with llama_context_ptr and drop the manual lla
webgpu: Handle unaligned writes in ggml_backend_webgpu_buffer_set_tensor (#29471) Fix: Handle unaligned writes in ggml_backend_webgpu_buffer_set_tensor Clang formatting Website: https://llama.app Atte
common : fix HF cache paths on Windows (#29475) Supersedes #29158 Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/atte
batch: migrate speculative, mtmd and server to batch_ext (#29385) adapt common add common_batch wip wip: spec cont common_speculative_process server_batch to use common_batch rm some stale calls Assis
ggml-openvino: mark unaligned batch-stride views unsupported (#29603) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50908876 macOS/iOS: macOS Apple Silico
models: pad on the left with ggml_pad_ext (#29567) models: pad on the left with ggml_pad_ext The Parakeet, LFM2-Audio, Granite Speech and Gemma 4 audio encoders build a left padding as a right pad fol
vulkan: include functional header (#29597) Fixes the compile error: no template named 'function' in namespace 'std' Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attes
server : support typed content (vision/audio/video) input for /v1/embeddings endpoint (#29556) server : support multimodal input for /v1/embeddings (Qwen3-VL-Embedding) Accept the OpenAI-style wrapped
mtmd: fix GCC 15 stringop-overflow in decode_embd_batch (#29607) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50949667 macOS/iOS: macOS Apple Silicon (ar
hexagon: support for backend sampler (#29502) hex-topk: trying to improve/cleanup the pipeline hex-sampling: add STEP op hex-sampler: add SUM op hex-sampler: update CPY to support sampling cases hex-b
hexagon: support tiled Q4_0 and Q8_0 GET_ROWS (#29511) hexagon: support tiled Q4_0 and Q8_0 GET_ROWS hex-get-rows: fix macros hex-get-rows: use tiled HVX dequantization Assisted-by: OpenCode hex-get-r
hrm : fix layer placement of z_l_init weight (#29512) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50522371 macOS/iOS: macOS Apple Silicon (arm64) macOS
llama-bench : fix OOB access of hf_file (#29515) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50528155
RPC: use RDMA completion channel to not spin (#29440) RPC: use RDMA completion queue to not spin add TODO for apple RDMA Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/
common : throw instead of abort on grammar without llguidance (#29516) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp
vulkan: fix argsort kernel selection for Adreno (#29469) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50544436 macOS/iOS: macOS Apple Silicon (arm64) mac
HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes (#28907) HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes CI: hip-quality-check: ignore spills for very
CUDA: tune fp16 tile FlashAttention configs for head sizes 40-112 (#26289) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50554486 macOS/iOS: macOS Apple S
sycl: FWHT kernels for block widths above 512 (#29243) The SYCL FWHT covers 64 to 512 via the standard butterfly network, plus 384/640/768/1280 via the Kronecker/Paley construction added separately in
opencl: refine bin kernel loading condition (#29503) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50560500 macOS/iOS: macOS Apple Silicon (arm64) macOS A
jinja : add support for dict builtin (#29477) add support for dict builtin add tests Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50563209 macOS/iOS: mac
common : make string_split throw on invalid values (#29518) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestatio
common : avoid side effects around params parsing (#29537) register --rpc unconditionally and call llama_supports_rpc() only from its handler print server 'initialization ...' log after args are parse
server : allow RANK pooling batch splitting for causal LLM rerankers (ie. Qwen3 and Qwen3-VL) (#28876) server : allow splitting RANK pooling for causal LLM rerankers Rerank models fall into two catego
vendor : update cpp-httplib to 0.58.0 (#29407) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50337762 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
hexagon: find software divide calls using binary inspection tool (#29449) hex-scripts: fix table alignment hex-scripts: find sw div calls using binary inspection tool Website: https://llama.app Attest
opencl: add A8 Q8_0 non-MoE dp4a binary kernel (#29439) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50382689 macOS/iOS: macOS Apple Silicon (arm64) macO
ggml-cpu: tiled mul_mat for k-quants (#27851) Added tiled mul_mat. For each mul_mat_one_chunk, quants are unpacked into (max) 256x256 tiles of int8, one routine per quent. Then microkernel computes 16
jinja : fix compile error (#29468) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50395469 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm6
jinja : implement sameas test (#29448) implement sameas test add tests Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50397675 macOS/iOS: macOS Apple Silic
Revert 'Change max context length for auto-fitting with unified KV (#28849)' (#29437) This reverts commit b04d4e5. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attest
server : fix wake_fd warning on Windows (#29479) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50455005
cuda: add F16 input to the FWHT (#29096) cuda: add F16 input to the FWHT The CUDA FWHT accepts F32 input only. This makes the source type a template parameter, so the kernel reads an F16 source direct
cuda: support Nemotron 3 Puzzle state size 96 for ssm scan (#28717) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50461180 macOS/iOS: macOS Apple Silicon
metal : fix graph capture and handle empty graphs (#29390) return early when the graph has no nodes drop the redundant reset of capture_compute: the decrement at the top of the function already transi
hexagon: use DMA for contiguous dim1 CONCAT (#29404) hexagon: use DMA for contiguous dim1 CONCAT Assisted-by: OpenCode hexagon: update CONCAT DMA for DMA64 Assisted-by: OpenCode Website: https://llama
hexagon: add q5_k quant type support (#29123) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50064037 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Si
CUDA: fuse RMS_NORM + SCALE into one kernel (#29393) #28068 builds the GDN q/k l2norm as ggml_scale(ggml_rms_norm(x, eps/n), 1/sqrt(n)). This adds 2 SCALE nodes per GDN layer, 96 extra kernel launches
musa: fix PH1 (MTT S5000) operator failures and build issues (#29193) musa: use 16-byte copies for MUSA like sm_70+ ggml_cuda_get_max_cpy_bytes() derives the copy width from CUDA_ARCH. mcc never defin
[SYCL] support sparse FA (#28796) fix conflict fix format issue rm unused code Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50188435 macOS/iOS: macOS App
rpc: include nb in the get_alloc_size cache key and floor the result at ggml_nbytes (#29283) rpc : include nb in the get_alloc_size cache key and floor the result at ggml_nbytes cont : remove redundan
HIP: bump HIP_VERSION requried for fp8 to avoid missing __hip_fp8_e4m3 support in 6.2 (#29231) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50216799 macO
llama : add llama_prec_policy + model-driven W4A4 path (#24364) Rebase and update based on #26675 Signed-off-by: ynankani ynankani@nvidia.com CI failure fix(launh_bounds overload on HIP) and cleanup S
metal : split fa kernels into per-dtype libraries (#29329) metal : split fa kernels into per-dtype libraries Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp cont : minor fix comment Website: ht