b10268
ci: fix pre-built binaries no longer working on macOS 15 and below (#26375) ci: fix pre-built binaries no longer working on macOS 15 and below ci: add macOS deployment target to disabled KleidiAI buil
Knowledge catalogue
ci: fix pre-built binaries no longer working on macOS 15 and below (#26375) ci: fix pre-built binaries no longer working on macOS 15 and below ci: add macOS deployment target to disabled KleidiAI buil
models : fix dflash wo_a reshape on load (#26577) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFrame
mtmd: support Qwen3-TTS (note: breaking change to llama-tts binary) (#26254) convert text model main model load ok convert encoder ok speaker encoder loading ok speaker enc graph adapt vocab for backb
sampler : remove 'full-context windows' from history-based samplers (#26524) Resolve -1 to 1024 instead of ctx-len for samplers Because of backend-sampling we initialize samplers before the complete l
mtmd: correcting duplicate empty audio chunks for short inputs (#26536) correcting duplicate empty audio chunks for short inputs tests.sh code restored Website: https://llama.app macOS/iOS: macOS Appl
server: decode Windows OEM output to UTF-8 in built-in tools (#26597) a child process writes in the OEM code page, which is not UTF-8 on a western Windows install, so accented output reaches the JSON
metal: implement DSv4 Lightning Indexer (#25893) metal: implement F16 Lightning Indexer Implement GGML_OP_LIGHTNING_INDEXER for 128-dimensional, 64-head inputs with F32 queries and weights plus F16 ke
llama : MTP support for DeepSeek V3.2 (#26457) llama : MTP support for DeepSeek V3.2 model : no need to include MTP layers during DeepSeek V3.2 model type discovery Co-authored-by: Stanisław Szymczyk
model: MTP support for Qwen3-Next (#25589) mtp for qwen3nex fix for python type-check Fix to compute num_mtp from directly mtp layer define opt_num_mtp_layers in _QwenMtpMixin and fix some comments Fi
server: add notice for upcoming default port change 8080 --> 9931 (#26508) server: add notice for upcoming default port change 8080 --> 9931 add link to PR correct to 9931 Website: https://llama.app m
CUDA: Fix data-races when reusing SMEM in block_reduce (#26385) CUDA: Fix data-races when reusing block_reduce block_reduce currently doesn't resync after reading from SMEM, causing potential data-rac
CUDA: Add backend sampler for penalties sampler (#25262) sampling: enhance penalty handling in common_sampler_init Set default value for penalty_last_n based on model context if not specified. Ensure
llama : allocate indexer cache only in 'full' indexer layers (#26474) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Appl
model: M3: Move MSA into a new memory implementation (#26338) Move MSA logic from llama-kv-cache into llama-kv-cache-msa cont : minor cont : ws fix Co-authored-by: Georgi Gerganov ggerganov@gmail.com
graph : fix unused input tensors in minimax m3 graph (#26519) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)
opencl: route large q6_K lm_head to the flat GEMV (#26427) add a direct size condition for large weights; the original dimension condition is insufficient -- q6_K lm_head for gemma-4 E2B has [1536, 26
arXiv:2607.28645v1 Announce Type: cross Abstract: Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshot
ggml-webgpu: add support for f16 repeat (#26307) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramew
model : load MiMo V2 MTP tensors only if used (#26412) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XC
sycl: fix classification of iGPUs (#26105) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Li
chat : add qwen3 specialized parser (#26252) Add tagged thinking tool parser chat : refactor and add permute helper cont : add support for <tool_call> omission cont : update tool delimiters cont : add
DeepseekV4 MTP + DSpark (#25784) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubunt
opencl: bugfix increment ref_count in ggml_backend_opencl_init() (#26162) Incrementing ref_count at the beginning is important later in the free() method of the ggml_backend_opencl_context at program
common: support the DSpark sidecar resolution (#26458) The dspark- files resolve like the other speculative sidecars: the -hfd tag applies to them, a requested sidecar resolves without a full model at
metal: implement DeepSeek V4 hyper-connections (#26459) Implement GGML_OP_DSV4_HC_COMB, GGML_OP_DSV4_HC_PRE, and GGML_OP_DSV4_HC_POST with SIMDgroup register and shuffle optimized kernels. Add Metal d
opencl: limit local workgroup size for GLU operation (#26383) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)
metal : add F16 support for bin ops (#26465) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework
metal : add SILU_BACK (#25982) feat(silu_back): implemented silu_back op for f32 fix(silu_back): removed redundant asserts in ggml-metal-ops.cpp function ggml_metal_op_silu_back. Website: https://llam
chat : enable tool call in thinking for DS4 (#26269) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFr
mtmd: add minicpmv46 downsample (#25993) add minicpmv46 downsample Signed-off-by: tc-mb tianchi_cai@icloud.com put downsample mode inside gguf. Signed-off-by: tc-mb tianchi_cai@icloud.com build mtmd_i
cli : persist reasoning_content in chat history (#26362) cli : persist reasoning_content in chat history llama-cli collected reasoning from the stream for display but only stored assistant content in
vendor : update BoringSSL to 0.20260730.0 (#26353) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFram
test: fix some CI errors (#26415) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubun
ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts (#25956) improve fa of quantized kv cache Fix some bugs and some comments. fix v type check and some comments Fix build error caus
sycl: fuse RMS_NORM + MUL (#26015) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubu
[SYCL] Support q2 mul_mat (#26231) support q2_0 in mul_mat support more q2_0 case Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABL
sycl : support dev2dev memcpy by DEV2DEV_MEMCPY_FORWARD (#26234) Co-authored-by: Neo Zhang Jianyu jianyu.zhang@intel.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S
ggml-zendnn : group matmul direct API for mul_mat_id (#25918) ggml-zendnn : group matmul API for mul_mat_id ggml-zendnn : scale MUL_MAT_ID fallback threshold by expert count Website: https://llama.app
llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized (#25871) llama : enforce the same K and V cache types for DeepSeek V4; enable FA if V cache is quantized
[SYCL] support the missed types in cpy (#26005) support the missed types in cpy use correct funct rm unused code Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (
SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc… (#25025) SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt processing fattn-mkl: fix interleaved dst layout in nor
cuda: extract Q2_0 elements via __byte_perm (#25603) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFr
server: correct accepted tokens when need draft token replay (#26320) spec: correct accepted tokens when need draft token replay cont : naming Co-authored-by: Georgi Gerganov ggerganov@gmail.com Websi
vulkan: update vulkan sdk to 1.4.357.0 (#26303) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramewo
llama : load MTP tensors only if they are really used (#26296) llama : load MTP tensors only if they are really used llama : skip loading MTP (if not used) in remaining models that support MTP Co-auth
Support rotated kv cache quant (#26180) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux
mtmd: add n_embd_head (#26342) Co-authored-by: Daniel Han unslothai@gmail.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED m
vulkan: Introduce driver version check for Windows Intel GPU to mitigate crashing (#25192) Removed crash guard for Intel Crash fixed from driver 32.0.101.8860 Added driver version check for windows Ch
vulkan: add POOL_1D op (#25431) vulkan : add pool1d push constants and pipeline field Declared data structures needed for POOL1D OP, which are the vk_op_pool1d_push_constants struct and pipeline_pool1
mimo2: address MTP review feedback (#26228) Co-authored-by: tnhnyc 115956684+tnhnyc@users.noreply.github.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm6
ggml : Fix issue with kleidiai ci and stringop overflow warning (#26277) Signed-off-by: Jonathan Clohessy Jonathan.Clohessy@arm.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) ma
metal: fix memory unwire if model is freed without any GPU operations (#26082) metal: fix memory leak if model is freed without any GPU operations metal: run dummy work only if residency sets are used
Remove custom cpu op from the M3 graph, express with stock ops (#26297) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS I
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu ar
ggml-cuda: Allow transpose-free gemmv computation (#26171) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. Website: https://llama.app macOS/iOS: mac
tests : avoid building get-model.cpp many times (#26317) tests : remove get-model.cpp tests : fix quant type selection Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sil
llama-context : sync pending async copies before clearing embd_seq (#25676) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED mac
Test support for alternative conv layout (#25617) add bool cwhn = true to conv_2d test cases add layout check at graph building time extend layout checks for conv2d.cu kernel in CPU back-end kernel ne
vulkan: Support quantized concat (#25684) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Lin
server: support inp embd to generate next token (#26313) server: support embd for sampled token fix ~server_batch() Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silico