b11512
model : fix DFlash output head sharing (#30111) llama : fix DFlash output head sharing Assisted-by: Codex dflash : read tied output weights from GGUF metadata Assisted-by: Codex llama : share tied wor
Knowledge catalogue
model : fix DFlash output head sharing (#30111) llama : fix DFlash output head sharing Assisted-by: Codex dflash : read tied output weights from GGUF metadata Assisted-by: Codex llama : share tied wor
CUDA: improve top-k algorithm selection (#28713) CUDA: radix top-k for large row counts Replaces CUB's per-row DeviceTopKKernel with a grid-over-rows radix select, gated on GGML_CUDA_TOPK_RADIX_MIN_RO
Musa FWHT fix (#30167) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54083178 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI
What's Changed server: proxy cloud usage and balance APIs by @drifkin in #18829 llama: fix clef head reads past 2GiB on windows by @Gigrise in #18777 cmd: remove account step from CLI onboarding by @h
What's Changed README: add oxi to community integrations by @maziluiosif in #18739 server: hide duplicate and downgrade guards from list by @dhiltgen in #18874 New Contributors @maziluiosif made their
When a legacy GGUF which requires our llama.cpp patch set is loaded, we do a lazy conversion on disk to make it llama.cpp compatible. To support downgrades, we temporarily hold both the new and old GG
ggml-webgpu: no dawn native features on wasi (#27069) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53445261 macOS/iOS: macOS Apple Silicon (arm64) macOS
sycl: add IQ3_S multi-column MMVQ (#29500) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53455527 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
vulkan: fix amd iGPU slow checkpoint read (#30049) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53465294 macOS/iOS: macOS Apple Silicon (arm64) macOS App
ggml-webgpu: fix flash_attn supports_op check for overlapping KV (#28205) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53548008 macOS/iOS: macOS Apple Si
imatrix : include clocale for std::setlocale (#30079) Co-authored-by: yarik dazzywi@github.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53551657 macO
vendor : update cpp-httplib to 0.60.0 (#30081) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53557265 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
musa: use the tile lightning indexer kernel (#30080) Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/535
vocab : add plamo fim tokens (#30090) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53567889 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (a
sampling : use greedy selection for eligible temperature-zero chains (#29797) sampling : use greedy selection for eligible temperature-zero chains Assisted-by: OpenAI Codex Apply suggestion from @gger
feat: add GLM5Next MTP, optimize (#29928) llama : add GLM5-Next NextN (MTP) graph Build the GLM5-Next multi-token-prediction head as graph_mtp: the NextN block embeds enorm(tok)+hnorm(h) through eh_pr
metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT (#30100) metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT ggml_metal_op_mul_mat_mma picks the residual of a fus
metal : few-row MMA mat-mul for the remaining src0 types (#30065) The generic few-row MMA kernel works for any type with a 16-weight dequantizer, so it now also takes BF16, Q1_0, Q2_0, MXFP4, Q2_K, Q3
llama: share the nextn tensor flags between models Follow-up of the TODO in glm5-next: move the trunk-only and MTP-only detection that each model copied into a nextn_flags helper of llama_model_base.
Developers aren’t becoming more careless; they’re being outpaced. The tools that let developers create more software should also take on more of the work of protecting it. The post Secret protection m
llama: fix k-pool scatter data race on shared sequences (#29994) llama: re-pool each shared k-pool rep once With shared cells every pool is re-pooled, and since the pooled keys are always scattered, t
ggml-openvino: skip unselected graph branches and support DUP Upstream #29622 adds a mixed token/embd branch to every input embedding graph through ggml_build_forward_select(). Its nodes are not flagg
test-llama-archs : initialize backends before generating models (#30034) With GGML_BACKEND_DL=ON, backends must be loaded explicitly before creating models. Assisted-by: Codex Website: https://llama.a
ggml: refactor selective expert copying to user code (#29943) ggml: refactor selective expert copying to user code tests: enroll two models into selective expert copy test tests: use deepseek2 as test
llama : re-reserve the sched when the nextn extraction flags change (#30020) The speculative MTP init enables NextN extraction on the target and draft contexts after both were created and their schedu
models : consolidate nextn row cropping into shared helpers (#30017) mimo2 : always emit h_nextn the other nextn-capable models set it unconditionally Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD mo
ggml: fix CLAMP on non-contiguous views (CPU, CUDA) (#29517) ggml: fix CLAMP on non-contiguous views (CPU, CUDA) CUDA clamped ggml_nelements values flat and ignored the view strides. CPU addressed row
RPC: add -sm tensor (#26610) rpc: allow -sm tensor fix flush for apple rdma move graph_uids to rpc_dispatcher cont : fix conflict cont: stop spinning dispatcher thread remove meta backend change add T
opencl: fix OOB read in adreno xmem GEMM (#30041) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53310534 macOS/iOS: macOS Apple Silicon (arm64) macOS Appl
model : add K2 Horizon dense and MoVA support (#29535) model: K2 Horizon gguf conversion code model: loading hparams and tensors in k2-horizon.cpp model: K2 Horizon compute graph model: K2 Horizon com
vocab : implement PLaMo-3 tokenizer pre-segmentation (#30045) vocab : implement PLaMo-3 tokenizer pre-segmentation The PLaMo-3 tokenizer inserts hard boundaries before running the Unigram DP, around <
cuda : add BF16 support for XIELU (#29955) The XIELU CUDA kernel template is already generic over the element type; only the F32/F16 type assertion and the else-if dispatch were missing. Add the nv_bf
The secondary button gets its own look back and the chat add button its own light surface. Pressable elements get a pointer cursor again, the font rendering smooths on the app shell, the root layout r
Implements the EmbeddingGemma2Model architecture on the MLX runner: 24-layer bidirectional text encoder with PLE, shared gemma4 vision/audio towers, mean-pool + L2 output. /api/embed accepts per-item
llama: support both embd + raw tokens in batch (#29622) llama: support both embd + raw tokens in batch add to test-llama-archs also check case llm_arch_supports_mixed_batch = false constant graph topo
log, server: self contained colors, split child commands from logs in router mode (#29895) log, server: make router child lines carry their own colors The logger writes the color reset after the trail
CUDA: prefer whole-tile FlashAttention scheduling for efficient two-stage kernels (#29435) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52707115 macOS/iO
CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size (#29633) CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size Signed-off-by: ynankani ynankani@nvidia.com adjust kernel selection l
metal : few-row MMA mat-mul (#29869) metal : few-row MMA mat-mul and batched copies for speculative decoding Speculative decoding verifies a few draft tokens per step. Without the tensor API, Metal ra
cuda: tile the lightning indexer over keys and tokens for 4 heads (#29901) cuda: tile the lightning indexer over keys and tokens for 4 heads With too few heads for a wmma tile, a block scores 64 keys
webgpu: add MMVQ support for Q1_0/Q5_0/Q5_1/Q3_K/Q5_K/Q6_K/MXFP4 (#29483) add supports for q1/q5/q3_k/q5_k/q6_k/mxfp4 of mmvq path Add K_QUANTS_HANDLING macro to q1_0 of mmvq path Website: https://lla
vulkan : Fix undeclared identifiers when -DGGML_VULKAN_RUN_TESTS=ON (#29912) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52752005 macOS/iOS: macOS Apple
ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 (#28479) ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 On the SpacemiT X60, IME matrix acceleration only covered Q4_0/Q4_1/Q4_K. Q8_0
kv-cache: fix restoring mismatched KV cache rotation by saving exact rotation metadata (#28498) kv-cache: save exact KV rotation metadata, reject restoring mismatched rotation tests : move the state r
llama : fix unexpected graph reallocation in the k-pool models (#29958) llama : fix unexpected graph reallocation in the k-pool models Both k-pool models built a graph shape that depends on state the
vulkan: sparse flash attention for quantized K/V (#29639) vulkan: sparse flash attention for quantized K/V Assisted-by: Claude vulkan: single-scan sparse FA index compaction The compaction ran one wor
vulkan: fix stale prealloc_y reuse across flash attention and soft_max (#29591) Assisted-by: Claude Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52808438
server: reject partial media truncation (#24076) server: reject partial media truncation server: keep only the keep_first fix Drop the mtmd test helper change, which no longer builds since clip_image_
CUDA: Optimize accumulation in mmq for NVFP4 type (#29857) ggml_cuda: optimize accumulation in mmq_vec_dot_fp4_fp4_mma for better performance remove whitespace fix: correct indentation in mma_block_sc
server: support vision input for Clef (#29969) server: support vision input for Clef move input_attn_causal to private extend old server_batch::embd server_batch::token::pos to multi dim nits fix abor
vulkan: fix Flash Attention shmem write out of bounds (#29988) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52864580 macOS/iOS: macOS Apple Silicon (arm6
llama.cpp : bump version to 0.6.0 (#29997) llama.cpp : bump version to 0.6.0 scripts : update summary prompt (#0) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attesta
hexagon: matmul and flash-atten scalability updates (#29974) hexagon: head-parallel flash_attn partitioning for row-split multicore In row-split mode each core computes its output row shard of every M
hexagon: add pool op support (#29995) hexagon: add pool_2d support hexagon: add pool_1d support hex-pool: dma changes hex-pool: Optimize HTP pooling boundaries and DMA pipelining hex-pool: code cleanu
We’re launching ReviewBench, a benchmark for code review agents built on representative GitHub pull requests, multi-source ground truth, calibrated evaluation, and production-aligned metrics. The post
Overview llama.cpp v0.6.0 introduces the new llama_batch_ext extended batch API (with llama_process) for mixed token/embedding inputs and MTP/deepstack state embeddings, adds support for the GLM-5.3-F
webgpu: add f16 support to fill/set_rows (#29897) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52497832 macOS/iOS: macOS Apple Silicon (arm64) macOS Appl
ci : pushing tag needs deploy key (#29937) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52545484 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
common : prepare load_from_models_dir() for path conversion (#29674) This is part of the fs::path modernization series. That was also the opportunity to remove fs_list(). Signed-off-by: Adrien Gallouë
spec : fix n-gram drafts rejected at temp > 0 after truncation (#29924) Co-authored-by: Pranesh Gonegandla pgonegandla@nvidia.com Website: https://llama.app Attestations: https://github.com/ggml-org/l