Search: “ui”

llama: support both embd + raw tokens in batch (#29622) llama: support both embd + raw tokens in batch add to test-llama-archs also check case llm_arch_supports_mixed_batch = false constant graph topo

log, server: self contained colors, split child commands from logs in router mode (#29895) log, server: make router child lines carry their own colors The logger writes the color reset after the trail

CUDA: prefer whole-tile FlashAttention scheduling for efficient two-stage kernels (#29435) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52707115 macOS/iO

CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size (#29633) CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size Signed-off-by: ynankani ynankani@nvidia.com adjust kernel selection l

metal : few-row MMA mat-mul (#29869) metal : few-row MMA mat-mul and batched copies for speculative decoding Speculative decoding verifies a few draft tokens per step. Without the tensor API, Metal ra

cuda: tile the lightning indexer over keys and tokens for 4 heads (#29901) cuda: tile the lightning indexer over keys and tokens for 4 heads With too few heads for a wmma tile, a block scores 64 keys

webgpu: add MMVQ support for Q1_0/Q5_0/Q5_1/Q3_K/Q5_K/Q6_K/MXFP4 (#29483) add supports for q1/q5/q3_k/q5_k/q6_k/mxfp4 of mmvq path Add K_QUANTS_HANDLING macro to q1_0 of mmvq path Website: https://lla

vulkan : Fix undeclared identifiers when -DGGML_VULKAN_RUN_TESTS=ON (#29912) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52752005 macOS/iOS: macOS Apple

ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 (#28479) ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 On the SpacemiT X60, IME matrix acceleration only covered Q4_0/Q4_1/Q4_K. Q8_0

kv-cache: fix restoring mismatched KV cache rotation by saving exact rotation metadata (#28498) kv-cache: save exact KV rotation metadata, reject restoring mismatched rotation tests : move the state r

llama : fix unexpected graph reallocation in the k-pool models (#29958) llama : fix unexpected graph reallocation in the k-pool models Both k-pool models built a graph shape that depends on state the

vulkan: sparse flash attention for quantized K/V (#29639) vulkan: sparse flash attention for quantized K/V Assisted-by: Claude vulkan: single-scan sparse FA index compaction The compaction ran one wor

vulkan: fix stale prealloc_y reuse across flash attention and soft_max (#29591) Assisted-by: Claude Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52808438

server: reject partial media truncation (#24076) server: reject partial media truncation server: keep only the keep_first fix Drop the mtmd test helper change, which no longer builds since clip_image_

CUDA: Optimize accumulation in mmq for NVFP4 type (#29857) ggml_cuda: optimize accumulation in mmq_vec_dot_fp4_fp4_mma for better performance remove whitespace fix: correct indentation in mma_block_sc

server: support vision input for Clef (#29969) server: support vision input for Clef move input_attn_causal to private extend old server_batch::embd server_batch::token::pos to multi dim nits fix abor

vulkan: fix Flash Attention shmem write out of bounds (#29988) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52864580 macOS/iOS: macOS Apple Silicon (arm6

llama.cpp : bump version to 0.6.0 (#29997) llama.cpp : bump version to 0.6.0 scripts : update summary prompt (#0) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attesta

hexagon: matmul and flash-atten scalability updates (#29974) hexagon: head-parallel flash_attn partitioning for row-split multicore In row-split mode each core computes its output row shard of every M

hexagon: add pool op support (#29995) hexagon: add pool_2d support hexagon: add pool_1d support hex-pool: dma changes hex-pool: Optimize HTP pooling boundaries and DMA pipelining hex-pool: code cleanu

arXiv:2610.02320v1 Announce Type: cross Abstract: Computer-use agents need to reliably ground action targets in complex desktop scenes, where multiple applications, overlapping windows, and visually s

Amazon SageMaker optimized generative AI inference introduces the aws-ai-ml skill through the Agent Toolkit for AWS, giving coding agents like Kiro, Claude Code, and Codex deep expertise in inference

webgpu: add f16 support to fill/set_rows (#29897) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52497832 macOS/iOS: macOS Apple Silicon (arm64) macOS Appl

ci : pushing tag needs deploy key (#29937) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52545484 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic

common : prepare load_from_models_dir() for path conversion (#29674) This is part of the fs::path modernization series. That was also the opportunity to remove fs_list(). Signed-off-by: Adrien Gallouë

spec : fix n-gram drafts rejected at temp > 0 after truncation (#29924) Co-authored-by: Pranesh Gonegandla pgonegandla@nvidia.com Website: https://llama.app Attestations: https://github.com/ggml-org/l

imatrix: calculate activation-based statistics for new format (GGUF) imatrices (#14891) Use activations to calculate the stats Determine calculation mode Compute entropy for activations Compute cosine

vulkan: fix rdna4 mat_vec tuning (#29934) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52579133 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silico

CUDA: fix MMQ memory fault if n_expert >> n_ubatch (#29941) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52581427 macOS/iOS: macOS Apple Silicon (arm64)

cuda : move blocks_per_col to where it is used (#29939) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/5

ci : set default permissions (#29945) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52595372 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (a

chat-peg-parser : clear current_tool when pending_tool_call is reset (#29942) A TOOL_ID node that arrives after TOOL_CLOSE wrote through current_tool, which still pointed into the just-destroyed pendi

ci : windows llvm build requires ninja multi-config (#29959) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52619546 macOS/iOS: macOS Apple Silicon (arm64)

cuda : move neu_padded to where it is used (#29940) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52636

ggml-cpu: support BF16/FP16/FP32 K tails in tinyBLAS on x86 (#29806) ggml-cpu: vectorize BF16 K tails in tinyBLAS tests: Skip tinyBLAS when use_ref is enabled so CPU tests compare against the vec_dot

Automated lint: 19 errors, 64915 warnings, 3 info

metal : add tensor API flash attention kernel for F16 KV (#29570) metal : add tensor API flash attention kernel for F16 KV cont : add tensor FA kernels for DK=DV=512 and DK=576, DV=512 cont : support

model: support nimble decision model (#29844) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52332773 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Si

ggml-cpu : fix soft_max_back wrong output when dst aliases src1 (#27096) ggml-cpu : fix soft_max_back wrong output when dst aliases src1 GGML_OP_SOFT_MAX_BACK is listed in ggml_op_can_inplace, so the

ggml-quants : avoid invalid rounding in qkx3 scale search (#29817) ggml-quants : avoid invalid rounding in qkx3 scale search The imatrix scale search can produce an infinite, NaN, or otherwise out-of-

spec : add probabilistic sampling for simple draft and MTP (#27694) Make the drafter probabilistic and the target verify by rejection sampling Drop stale spec_draft_q before drafting Fallback to argma

CUDA: fuse shared experts into MMVQ (#29184) CUDA: fuse shared experts into MMVQ check if buffer is null move stride_col_dst to fusion args Website: https://llama.app Attestations: https://github.com/

model: add support for clef decision model (text-only) (#29831) init support for clef (text only) more static graph clean up nits nits 2 Update gguf-py/gguf/constants.py Co-authored-by: Sigbjørn Skjær

qwen4exp : halve the indexer score memory (#29825) qwen4exp : halve the indexer score memory The indexer scored all heads in one product and rectified a copy of it, so two [n_pool, n_idx_h, n_tokens]

ggml-openvino: update to 2026.4.1, optimize performance, expand ops, improve device listing. (#29852) ggml-openvino : Qwen3.5 MoE perf (#312) Squash of ravi9#312: ggml-openvino: add detailed inference

graph: gather the recurrent states once so the reserve covers every split (#29856) build_rs gathered the extra states (n_rs - n_seqs rows) with their own get_rows. The worst-case reserve has n_rs == n