b11551
model : support MiniCPM-V 4.7 (#29416) mtmd : add MiniCPM-V 4.7 support Signed-off-by: tc-mb tianchi_cai@icloud.com model : allow mrope time from an extra position slot Signed-off-by: tc-mb tianchi_ca
Knowledge catalogue
model : support MiniCPM-V 4.7 (#29416) mtmd : add MiniCPM-V 4.7 support Signed-off-by: tc-mb tianchi_cai@icloud.com model : allow mrope time from an extra position slot Signed-off-by: tc-mb tianchi_ca
server: leave a busy slot untouched when a request pins it (#30295) A request asking for a busy id_slot still ran the prompt cache update on that slot before being deferred. When the RAM cache held a
Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted
hexagon: fix IM2COL patch-embed DMA ring overflow (#30189) hexagon: fix IM2COL patch-embed DMA ring overflow The exact-tiling (stride == kernel, no pad/dilation) IM2COL DMA kernel issues ICKH DDR->VTC
CUDA: pass src1 precision to host MMQ config helpers (#30168) CUDA: pass src1 precision to host MMQ config helpers Signed-off-by: ynankani ynankani@nvidia.com make src1 prec explicit Signed-off-by: yn
metal: add the 128/96 flash attention kernels, accept every compiled pair (#30209) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54260937 macOS/iOS: macOS
server: use port 9931 by default (#30159) server: use port 9931 by default revert unrelated changes Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54269123
ci: fix Models Backend webgpu by supporting GGML_OP_DUP (#30216) The mixed batch path of PR-29622 writes the token rows with set_rows into a dup of the embeddings. WebGPU did not support DUP, so the d
musa: drop mp_21 from the default architectures (#30203) musa: drop mp_21 from the default architectures Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Address review comments Signed-off-by: Xiao
webgpu: use 2D workgroup dispatch for all the ops which use 1D dispatch (e.g., rms_norm) (#30219) remove 1D workgroups dispatching formatting Website: https://llama.app Attestations: https://github.co
meta : handle host views (#30217) meta : handle views of tensors allocated on the host A view shares the memory of its view_src, so ggml-alloc never allocates a view in the buffer of the split it land
ggml: fix OOB write in ggml_acc with negative offset (#30135) ggml_acc_impl narrowed a size_t offset to int32_t without checking that it fits, so a large offset could truncate to a negative int32_t. T
llama: keep the backend sampling graph static across ubatches (#30223) The reserve builds n_outputs_max_per_seq sampling chains per sampler, while a decode built one per output row, so the graph chang
model : use exact GELU for ModernBERT encoders (#30108) model : use exact GELU for ModernBERT encoders Assisted-by: Codex model : keep tanh GELU aliases on ggml_geglu Assisted-by: Claude Opus 5.5 mode
opencl: fix kernel compilation for a6x GPUs (#30176) opencl: skip kernel_cpy_f32_f32_pack on A6X to avoid shader compiler crash The A6x compiler backend found in iot device with a623 (E031.50.31.01) c
CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807) CUDA: fuse copy of updated state snapshots into recurrent cache with ssm_scan CUDA: remove redundant cuda copies with K==1 (non spec-dec) sce
llama-bench: respect -fitc if bigger than required benchmark size (#28331) Assisted-By: opencode,llama.cpp,Qwen3.6-35B-A3B,Qwen3.8-27B Website: https://llama.app Attestations: https://github.com/ggml-
graph : reorder get_rows for embeddings (#30160) graph : reorder get_rows for embeddings cont : fix gemma4 and improve input embedding construction logic cont : add TODO for lora cont : fix raw embedd
llama : add a GPU cache for MoE experts kept in host memory (#29887) llama : add a GPU cache for MoE experts kept in host memory Assisted-by: Claude use llama_moe_cache_ptr Website: https://llama.app
mtmd: add cohere2 vision support (#30062) cohere2 vision model address comments remove redundant mapping follow existing patterns fused linear_1 Website: https://llama.app Attestations: https://github
cuda: FWHT kernels for block widths above 512 (#29100) The CUDA FWHT covers widths 64 to 512. It runs one row per warp and keeps N/32 values per lane, so wider blocks need more registers per lane than
model : add LiquidAI/d1-3B decision model (#30110) model : add LiquidAI/d1-3b decision model mtmd : read LFM2 image resize algo from GGUF Assisted-by: Claude Opus 5.5 common : rename decision type d1
chat : name tool and argument parser rules by index (#30088) bugfix: infinite recursion caused by a tool named 'call'(#29967) chat : use index for schema and argument rules tests : remove tests tests
chat: fix jinja parser for TranslateGemma (#30096) chat: fix jinja parser for translategemma log if missing soruce_lang_code or target_lang_code Website: https://llama.app Attestations: https://github
hexagon: support tiled Q4_K and Q6_K GET_ROWS (#30115) hexagon: support tiled Q4_K GET_ROWS Assisted-by: OpenCode properly reject Q4_K views Assisted-by: OpenCode hexagon: support tiled Q6_K GET_ROWS
model : add LiquidAI/d1-omni-600M decision model (#30114) model : add LiquidAI/d1-omni-600M decision model Assisted-by: Claude Opus 5.5 mtmd : keep conformer GLU sigmoid on CUDA Assisted-by: Claude Op
hexagon: Q6_K weight dequant speedup (#30121) hexagon: Q6_K weight dequant speedup Assisted-by: OpenCode unroll by another factor of 2 Assisted-by: OpenCode Website: https://llama.app Attestations: ht
hexagon: enable alloc_buffer_n (#30126) hex-bufs: add support for alloc_buffer_n hex-bufs: add support for splitting large tensors into separate buffers hex-bufs: update GGML_HEXAGON_MBUF to accept th
sycl: remove duplicate block-size defines from op headers (#29507) All were duplicates of the values in ggml/src/ggml-sycl/presets.hpp Website: https://llama.app Attestations: https://github.com/ggml-
sycl: add grouped MoE XMX GEMM (#29245) Co-authored-by: cwriter cwriter@localhost Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53864714 macOS/iOS: macOS
sycl: fuse the delta-net alpha gate (add + unary + mul) (#29687) sycl: fuse the delta-net alpha gate (add + unary + mul) tests: cover the fused add + unary + mul chain sycl: give the fused alpha gate
CUDA: fix norm family kernels when ne[2]/ne[3] exceed grid dim limits (#28175) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53919465 macOS/iOS: macOS App
cuda : Use byte strides for roll to allow non-contiguous ROLL operations (#29547) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53926988 macOS/iOS: macOS
vendor : update cpp-httplib to 0.60.1 (#30134) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53949333 m
vulkan : fix TOP_K for +inf/NaN inputs and k = 1 on negative values (#30107) The bucket search in topk_nary_search.comp started from the range [0, 0xFF800000), which ends just below the ordered-uint m
CUDA: fix CCCL version guard breaking on major version rollover (#29453) CUDA: fix CCCL version guard breaking on major version rollover The guard compared the major and minor components independently
model : fix DFlash output head sharing (#30111) llama : fix DFlash output head sharing Assisted-by: Codex dflash : read tied output weights from GGUF metadata Assisted-by: Codex llama : share tied wor
CUDA: improve top-k algorithm selection (#28713) CUDA: radix top-k for large row counts Replaces CUB's per-row DeviceTopKKernel with a grid-over-rows radix select, gated on GGML_CUDA_TOPK_RADIX_MIN_RO
Big new paper nobody is talking about ‼️ 🌎 Robot world models now have scaling laws. Meta trained an 8B-parameter JEPA on 15,000 hours of robot video to prove it. RoboJEPA (FAIR at Meta + Mila) imagin
I've been tinkering with GLM-5.3-Flash on two 64GB CMP 170HX cards for a while, and the setup is finally stable enough that I figured I'd share it. I also compared it against the Qwen3.8-Flash-Next se
imatrix : include clocale for std::setlocale (#30079) Co-authored-by: yarik dazzywi@github.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53551657 macO
musa: use the tile lightning indexer kernel (#30080) Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/535
sampling : use greedy selection for eligible temperature-zero chains (#29797) sampling : use greedy selection for eligible temperature-zero chains Assisted-by: OpenAI Codex Apply suggestion from @gger
feat: add GLM5Next MTP, optimize (#29928) llama : add GLM5-Next NextN (MTP) graph Build the GLM5-Next multi-token-prediction head as graph_mtp: the NextN block embeds enorm(tok)+hnorm(h) through eh_pr
metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT (#30100) metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT ggml_metal_op_mul_mat_mma picks the residual of a fus
metal : few-row MMA mat-mul for the remaining src0 types (#30065) The generic few-row MMA kernel works for any type with a 16-weight dequantizer, so it now also takes BF16, Q1_0, Q2_0, MXFP4, Q2_K, Q3
llama: fix k-pool scatter data race on shared sequences (#29994) llama: re-pool each shared k-pool rep once With shared cells every pool is re-pooled, and since the pooled keys are always scattered, t
test-llama-archs : initialize backends before generating models (#30034) With GGML_BACKEND_DL=ON, backends must be loaded explicitly before creating models. Assisted-by: Codex Website: https://llama.a
ggml: refactor selective expert copying to user code (#29943) ggml: refactor selective expert copying to user code tests: enroll two models into selective expert copy test tests: use deepseek2 as test
llama : re-reserve the sched when the nextn extraction flags change (#30020) The speculative MTP init enables NextN extraction on the target and draft contexts after both were created and their schedu
models : consolidate nextn row cropping into shared helpers (#30017) mimo2 : always emit h_nextn the other nextn-capable models set it unconditionally Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD mo
ggml: fix CLAMP on non-contiguous views (CPU, CUDA) (#29517) ggml: fix CLAMP on non-contiguous views (CPU, CUDA) CUDA clamped ggml_nelements values flat and ignored the view strides. CPU addressed row
model : add K2 Horizon dense and MoVA support (#29535) model: K2 Horizon gguf conversion code model: loading hparams and tensors in k2-horizon.cpp model: K2 Horizon compute graph model: K2 Horizon com
vocab : implement PLaMo-3 tokenizer pre-segmentation (#30045) vocab : implement PLaMo-3 tokenizer pre-segmentation The PLaMo-3 tokenizer inserts hard boundaries before running the Unigram DP, around <
cuda : add BF16 support for XIELU (#29955) The XIELU CUDA kernel template is already generic over the element type; only the F32/F16 type assertion and the else-if dispatch were missing. Add the nv_bf
llama: support both embd + raw tokens in batch (#29622) llama: support both embd + raw tokens in batch add to test-llama-archs also check case llm_arch_supports_mixed_batch = false constant graph topo
log, server: self contained colors, split child commands from logs in router mode (#29895) log, server: make router child lines carry their own colors The logger writes the color reset after the trail
CUDA: prefer whole-tile FlashAttention scheduling for efficient two-stage kernels (#29435) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52707115 macOS/iO
CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size (#29633) CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size Signed-off-by: ynankani ynankani@nvidia.com adjust kernel selection l
metal : few-row MMA mat-mul (#29869) metal : few-row MMA mat-mul and batched copies for speculative decoding Speculative decoding verifies a few draft tokens per step. Without the tensor API, Metal ra