b11126
vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415) vulkan: optimize IQ4_XS matmul kernels Assisted-by: OpenAI Codex vulkan: address IQ4_XS review nits drop the dead LOAD_VEC_A != 8 branch in the IQ4_X
Knowledge catalogue
vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415) vulkan: optimize IQ4_XS matmul kernels Assisted-by: OpenAI Codex vulkan: address IQ4_XS review nits drop the dead LOAD_VEC_A != 8 branch in the IQ4_X
model : support Gemma4 DSpark draft backbone (#29226) dspark: add Gemma 4 draft support Add GGUF conversion and runtime support for full-attention and SWA Gemma 4 DSpark drafts, including tied output
server: Dedup the draft HF model via dedup-cache-models (#27934) server: Dedup the draft HF model via dedup-cache-models Fixes #27846 server: avoid capturing structured binding in lambda Website: http
server: accept OpenAI video_url content type and data: video URIs (#27921) The OpenAI chat completions API specifies content part type 'video_url' with a {'url': ...} object, and clients typically sen
jinja : parse unary +/- before variables (#29244) jinja : parse unary +/- before variables Lexer already emits unary_operator for -n / +n, and runtime executes unary -. Parse them at multiplicative pr
server: fix token counting API crash on sleep (#29309) server: wake up sleeping server correctly server: wake up sleeping server correctly (local aliases removed) Website: https://llama.app Attestatio
CUDA: enable sparse-fa for dsv4 prefill (again) (#29298) CUDA: enable sparse-fa for dsv4 prefill (again) CUDA: unroll the query loop of the sparse mask scan The query loop of flash_attn_mask_to_sparse
tests: add -b/--backend option to test-llama-archs for testing a specific backend (#27372) tests: add backend option to test-llama-archs Update tests/test-llama-archs.cpp Co-authored-by: Johannes Gäßl
arXiv:2609.23907v1 Announce Type: new Abstract: Generative models of neural activity could help characterize tissue dynamics, compare experimental conditions, and simulate population activity for appl
vendor : update cpp-httplib to 0.57.1 (#29239) Signed-off-by: Adrien Gallouët adrien@gallouet.fr Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49092442 ma
hexagon: new HMX-optimized GATED_DELTA_NET (#29199) hex-gdn: start putting together HMX support for GDN hex-gdn: working hmx but not-pipelined and slow for now hex-gdn: re-write vtcm layout handling a
ci : publish snapdragon builds in release workflow (#29007) The snapdragon CI builds packages only to feed the QDC device tests, so Hexagon NPU binaries never reached the releases page. Build both tar
chat : Fix Muse Glimmer tool-call first parser error (#29242) Fix Muse Glimmer tool-call first parser error Add test to verify Condense patterns remove test for trigger patterns Website: https://llama
convert: add MiMo-V2.6 support (#29257) convert: add MiMo-V2.6 support Hoist the K3 mxfp4 conversion repack into base.py so it can be reused Remove decoder from mmproj convert Update conversion/mimo.p
spec : support DFlash for HunyuanOCR (#28890) model : add DFlash layer-input taps for HunyuanVL DFlash speculative decoding needs the target graph to expose the residual stream entering each layer (re
server: Add support for binding to multiple addresses (#28690) Add support for binding llama-server to multiple addresses Assisted-by: Codex remove redundant thread handler make it clear about overlap
metal : gate mul_mm_id src1 rescale behind ggml_prec (#29029) metal : gate mul_mm_id src1 rescale behind ggml_prec Assisted-by: Claude Fable 5.1 ggml-webgpu: reject MUL_MAT_ID when src1 precision is F
vulkan: add Intel Xe flash attention optimization kernels (2/3, Xe-LPG Plus/Xe2/Xe3) (#24406) vulkan : Intel FA kernel optimization for split k path vulkan : Host code update for Intel split k FA kern
server: support input_image in function_call_output (#20663) (#22575) server: support input_image in function_call_output (#20663) server: fix if statement spacing server: avoid repeated type lookup W
server: fix router eviction races with the existing queue (#29217) server: route every model load through the queue A model loaded by the fast path has no queue entry, so tick() evicts it at its LOADE
opencl: add bin kernel kernel_gemm_noshuffle_q4_k_q8_1_dp4a_ila_a8_bin (#29056) opencl: add A8 Q4_K non-MoE dp4a binary kernel opencl: rename binary kernel selection helpers Website: https://llama.app
NVIDIA Confidential Computing (CC) secures large‑language‑model inference by providing encrypted virtual machines, GPUs, and NVLink, enabling processing of sensitive data in trusted hardware. Running
arXiv:2609.24906v1 Announce Type: new Abstract: Dormant tree pruning is labor-intensive yet essential for maintaining modern high-productivity fruit orchards. In this work, we focus on pruning of mode
cuda : tune MMVQ to MMQ crossover for SM70 (Volta) (#28912) tune MMVQ to MMQ crossover for SM70 (Volta) Signed-off-by: Yangyu Chen cyy@cyyself.name Apply suggestion from @JohannesGaessler Apply sugges
hexagon: overhaul of buffer and DMA handling to support 64bit mappings + improvements (#29197) hex-dma64: enable support extended buffer mappings and 64bit dma hex-dma64: expand binary ops to support
sycl : coalesce MKL-FA softmax loads instead of one work-item per row (#28918) sycl : coalesce MKL-FA softmax loads instead of one work-item per row better human readable variable name Website: https:
json: Fixed json enum handling (#28518) Fixed json enum handling Added common_json_value handling for enum values. Added tests/test-json.cpp to cover testing of some aspects of common_json. Removed te
ggml-metal : simplify fusion pattern op list declaration (#29206) ggml-metal : derive non-empty fusion ops from ops_all Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp ggml-metal : drop _all su
server : do not forward --api-key-file to router-spawned child instances (#28938) In router mode, authentication belongs to the router. unset_reserved_args() already unset LLAMA_API_KEY, but did not u
args: add env vars for temperature, top-p, min-p and penalties (#27380) Allow configuring --temp, --top-p, --min-p, --repeat-penalty, --presence-penalty and --frequency-penalty via LLAMA_ARG_* so llam
tests/test-backend-ops : allow regex entries in the -o filter (#29204) tests/test-backend-ops : allow regex entries in the -o filter so far -o only accepted a comma separated list of exact op names or
test-llama-archs : make tensor data stdev configurable and improve help (#29133) test-llama-archs : make tensor data stdev configurable Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp test-llam
cuda: fix sm_70 tile compilation error (#29224) The 5-argument load_ldmatrix added in 1884824 only defines tile<16,8>, so the Volta tile<8,4> does not match. See #29222 for details. Building on 188482
chat : add dedicated Ling 3.0 (Bailing V3) parser (#28682) chat: add dedicated Ling 3.0 (Bailing V3) parser Ling 3.0 Flash templates pre-open the think block in the generation prompt, so the model nev
metal: add F16 input to the FWHT (#29094) metal: add F16 input to the FWHT The Metal FWHT kernel accepts F32 input only. This change makes the source type a template parameter, so the kernel reads an
fix(mamba) : make time-step projection input contiguous (#28832) mamba : make time-step projection input contiguous Assisted-by: ChatGPT mamba : skip contiguous copy after normalization Assisted-by: C
common/peg : handle invalid utf-8 sequences in the AST (#29161) common/peg : handle invalid utf-8 sequences in the AST cont : return maximal subpart per Unicode recommendations cont : remove strict ar
metal : support arbitrary hc in dsv4_hc_pre (#29169) the dsv4_hc_pre kernels hardcoded hc = 4 via a constexpr used with simd_shuffle, so the op was rejected by supports_op for any other hc and fell ba
So this jev thingy is getting kind of big... tbh it seems overhyped by a large margin, but here we are. Not that its bad, just feels like we usually ignored larger things... Anyway to the point of thi
cuda : fix CUB argsort corruption caused by in-place keys (#28389) argsort_f32_i32_cuda_cub called the one-shot DeviceRadixSort::SortPairs API with d_keys_in == d_keys_out (temp_keys, temp_keys). CUB'
metal : support qwen4exp hc ops (#29000) Add support for the new DSV4 HC op variants used by qwen4exp: hc_pre with per-element sigmoid gate (gated variant) hc_post with identity mixing (comb == nullpt
test-llama-archs : generate dummy test vocab (#29084) Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations
server : improve startup log messages (#29125) server-models : show source per model in log Show [source] tag (preset/models_dir/cache) per model instead of cryptic * marker Show HF hub cache path in
hexagon: enable support for TOP_K op (#29113) hexagon: enable support for TOP_K op hex-topk: thread single-row TOP_K, raise VTCM-based size cap hex-topk: fix TOP_K mdev row partitioning hex-topk: opti
vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 1024 experts (#28501) vulkan: raise the hoisted row-id limit for mul_mat_id to 512 experts The expert-count shader (count_experts.comp
cmake : fix build when GGML_CPU=OFF and GGML_CUDA=ON (#29026) fix: build fails when GGML_CPU=OFF and GGML_CUDA=ON fix: eol in examples/convert-llama2c-to-ggml/CMakeLists.txt file Website: https://llam
vocab : add ufakzeka pre-tokenizer (#29033) vocab : add ufakzeka pre-tokenizer vocab : move ufakzeka to the models list and regenerate the hash mapping Website: https://llama.app Attestations: https:/
vulkan: add IQ3_S MMQ matmul kernels (#28822) vulkan: add IQ3_S MMQ matmul kernels Make block_a_to_shmem do 2-byte loads (110 bytes is divisible by 2) Align the check, IQ3_S is also using K tile size
ggml-webgpu: fix supports_op condition for GET_ROWS (#28978) fix get_rows vec4 handling Add src strides checking to vec4_aligned of get_rows and the new test case. Website: https://llama.app Attestati
Model-Saver: Write the SWA pattern, 15 more architectures roundtrip (#29042) llama: read the SWA pattern as a period or a per-layer array Add llama_model_base::load_swa_pattern(), which reads sliding_
ggml : check for allocation failures to prevent crashes (#28149) ggml : check for allocation failures to prevent crashes wording Website: https://llama.app Attestations: https://github.com/ggml-org/ll
opencl: add bin kernel kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin (#28678) opencl: add A8 Q6_K non-MoE binary kernel opencl: fix layout compatibility Website: https://llama.app Attestations:
hexagon: HMX flash-attention head_dim padding (support DK=DV=72) (#26539) Allow HMX flash-attention to run with head_dim not a multiple of 64 (e.g. SigLIP head_dim=72), by operating on DK/DV rounded u
hexagon: im2col update (#29103) ggml-hexagon: accept 1D and padded IM2COL ops ggml-hexagon: make pure-DDR IM2COL kernel is_2D-aware ggml-hexagon: extend IM2COL DMA patch-embed fast path to 1D ggml-hex
opencl: add support for bin kernel flash_attn_f32_f16_bin (#29046) opencl: add flash_attn_f32_f16_bin opencl: guarded prefill fa Website: https://llama.app Attestations: https://github.com/ggml-org/ll
arXiv:2609.20747v1 Announce Type: cross Abstract: Reinforcement learning constitutes a promising approach owing to its potential for superhuman performance and self-learned policies. However, its appl
[SYCL] Fix function signature for ggml_backend_sycl_split_buffer_type (#28981) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48082811 macOS/iOS: macOS App
vulkan: support qwen4exp hc ops (#28988) vulkan: support qwen4exp hc ops fix stale comment [no-ci] Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48109146
vulkan: fix buffer_reference alignment in im2col shaders (#28996) Both im2col.comp and im2col_3d.comp declare D_ptr without an explicit buffer_reference_align, so glslang emits writes through it as Al
opencl: fix various warnings (#28984) opencl: fix warnings opencl: fix warnings for non adreno Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48132480 macO