All entries

mtmd : fix deprecated strdup warning on Windows (#29863) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/

Learn three ways to get noticed and grow your career as AI reshapes how developers build software. The post AI is rewriting the developer career ladder. Here’s how to stand out. appeared first on The

Qwen4Exp: add MTP (#29761) Qwen4Exp: add MTP remove has_state member, check via ctx_bufs being non-empty consistent naming + less verbose comments cont : clean-up recurrent memory cont : clean-up comm

CUDA: Handle compute type for NVFP4 on cublass path (#29173) CUDA: Handle compute type for NVFP4 on cublass path Signed-off-by: ynankani ynankani@nvidia.com Use BF16 compute type for quantized models

llama : fix invalid assert in recurrent memory (#29799) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52014664 macOS/iOS: macOS Apple Silicon (arm64) macO

webgpu: add bfloat16 support for MUL_MAT/MUL_MAT_ID/GET_ROWS- #29358 (#29358) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52019658 macOS/iOS: macOS Appl

metal : release temporary private transfer buffers (#29777) metal : release temporary private transfer buffers Assisted-by: OpenAI Codex metal : fix order and formatting Co-authored-by: Niklas Wenzel

cuda : route sm70 to the Turing MMVQ nwarps table (#29753) cuda : route sm70 to the Turing MMVQ nwarps table Volta (sm_70) has no MMVQ parameter table of its own and falls through to GENERIC, which la

server: return HTTP 400 for invalid embedding requests (#29060) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52037093 macOS/iOS: macOS Apple Silicon (arm

hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA (#29685) hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA hex-cpy: various fixes on top of the co

llama : clamp kpool re-pool bound to existing pools (#29805) tests : simplify function signature llama : clamp kpool re-pool bound to existing pools The n_tokens/kpool + n_seqs_unq bound on n_new_g ov

common,rpc : fix cache dir creation through symlinks on buggy libstdc++ (#29816) See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101510 Close #29759 Signed-off-by: Adrien Gallouët angt@huggingface.co

CUDA: fix 2 broken Volta FA cases (#29803) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52093363 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic

hexagon: add q2_k and q3_k quant type support (#29717) hexagon: add q2_k and q3_k quant type support hex-qk: consistent allocation of src1_row_size Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.co

qwen4exp: fix tests (#29819) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52169567 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, Kle

hexagon: install rebuilt HTP skels (#29828) hexagon: install rebuilt HTP skels Assisted-by: OpenCode hexagon: fix HTP skel catalog dependencies Assisted-by: OpenCode Website: https://llama.app Attesta

vulkan: add logging to pipeline compile issues (#29794) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52203370 macOS/iOS: macOS Apple Silicon (arm64) macO

ggml : add alloc_buffer_n to buffer type interface (#23671) ggml : add alloc_buffer_n to buffer type interface Add alloc_buffer_n method to ggml_backend_buffer_type_i interface, with a public API ggml

qwen4exp : optimize mask constructions (#29824) qwen4exp : optimize mask constructions cont : apply the same change for GLM5-next Website: https://llama.app Attestations: https://github.com/ggml-org/l

vulkan: disable large matmul tile on Samsung GPUs with 32KB shared memory (#28531) Assisted-by: Claude Opus Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/

llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) (#29818) init conversion convert: ok model loaded add server code improve conversion script support shared prompt prefix

llama : preserve original batch order for speculative decoding layer inputs (#29019) llama: preserve original batch order for layer inputs Assisted-by: Codex tests: cover layer-input order across KV l

args: fix cli download mmproj arg (#28977) tests: add tests for cli download arg parsing args: fix cli download mmproj arg Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cp

Hexagon: optimize ALLREDUCE with support for safe scatter mode (#29757) hex-allreduce: add support for safe scatter mode hex-allreduce: pare down excessive comments hex-allreduce: re-write to remove r

cuda: guard the iq4_nl dequantize row kernel against short rows (#29683) dequantize_block_iq4_nl writes QK_K values per block, but a row can be shorter than that (an IQ4_NL row is only guaranteed to b

ggml-opencl : replace alloca() with std::vector (#29765) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/

webgpu: fix SSM_SCAN binding aliasing (#29750) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51685637 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S

model : re-enable -sm tensor for qwen4exp (#28569) #27941 disabled -sm tensor for qwen4exp because test-llama-archs asserted on the Meta device once the fixture carried a PLE layer: GGML_ASSERT(ggml_b

metal : use bf16 math for mxfp4 mul-mat (#29770) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51752863 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple

llama-bench : fix verbosity filter to show GGML_LOG_ERROR (#28229) bench : fix verbosity filter to show GGML_LOG_ERROR (#28107) bench: remove dead variables Website: https://llama.app Attestations: ht

vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3 (#29734) vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3 The original tokenizer configs for PLaMo-2 and PLaMo-3 have add_bos_token: true a

opencl: mark vec subgroup bcast as supproted for Adreno E17 compiler (#29698) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51780568 macOS/iOS: macOS Appl

common : add LLM-jp-4.1 Harmony dialect handler (#29681) LLM-jp-4.1 uses the GPT-OSS format, but its tokenizer decodes a space after every special token and parallel tool calls are separated by <|end|

BLAS : Document AOCL-BLAS build and label the device AOCL-BLAS (#29640) BLAS : Document AOCL-BLAS build and label the device AOCL-BLAS AOCL-Blas : Add an AOCL-BLAS Quick Start and drop the fixed versi

hex-workqueue: fix race condition in seqn getting out of sync with idx_read/write (#29785) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51906249 macOS/iO

HIP: avoid treating CDNA as dgx spark for gqa_ratio 20 in fattn_mma dqk 576 (#29572) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51919562 macOS/iOS: mac

llama-mmap : avoid a second full-size copy of each tensor with direct-io (#29749) Assisted-by: Claude Co-authored-by: Pranesh Gonegandla pgonegandla@nvidia.com Website: https://llama.app Attestations:

jinja : skip copying loop scope unless a loop filter needs it (#29776) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51940374 macOS/iOS: macOS Apple Silic

meta: clear inactive AllReduce shards with FILL, not SCALE (#29793) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51949614 macOS/iOS: macOS Apple Silicon

mtmd: cap max_image to ubatch for non_causal models (#29773) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51957111 macOS/iOS: macOS Apple Silicon (arm64)

vulkan: Tune GDN kernel, fix Intel performance (#29476) vulkan: tune GDN shader tune for Intel Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51306598 macO

opencl: fix get_tensor for q5_K adreno gemm_nonshuffle kernel (#29555) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51311810 macOS/iOS: macOS Apple Silic

ci: add zdnn backend build but not test (#29541) ci: add zdnn backend build but not test Signed-off-by: Aaron Teo aaron.teo1@ibm.com ci: attempt to run a ubuntu 26.04 container Signed-off-by: Aaron Te

ggml-cuda: HIP: optimize packed byte subtraction (__vsubss4 -> __vsub4) (#29478) ggml-cuda: HIP: optimize non-saturating packed byte subtraction (__vsubss4) CI: ignore 1 spilled vgpr in fattn_vec Co-a

hexagon: optimize concat op (#29673) hex-concat: reduce pkts in gather/transpose hot loop gather directly into dst buffer, use special instruction for gather sync hex-concat: use fastdiv replace calls

model : support classifier_pooling for rerankers (#29627) model : support classifier_pooling for ModernBERT rerankers Assisted-by: Claude Opus 5.5 model : read classifier pooling type in load_hparams