All entries

Support rotated kv cache quant (#26180) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux

mtmd: add n_embd_head (#26342) Co-authored-by: Daniel Han unslothai@gmail.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED m

vulkan: Introduce driver version check for Windows Intel GPU to mitigate crashing (#25192) Removed crash guard for Intel Crash fixed from driver 32.0.101.8860 Added driver version check for windows Ch

vulkan: add POOL_1D op (#25431) vulkan : add pool1d push constants and pipeline field Declared data structures needed for POOL1D OP, which are the vk_op_pool1d_push_constants struct and pipeline_pool1

mimo2: address MTP review feedback (#26228) Co-authored-by: tnhnyc 115956684+tnhnyc@users.noreply.github.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm6

ggml : Fix issue with kleidiai ci and stringop overflow warning (#26277) Signed-off-by: Jonathan Clohessy Jonathan.Clohessy@arm.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) ma

metal: fix memory unwire if model is freed without any GPU operations (#26082) metal: fix memory leak if model is freed without any GPU operations metal: run dummy work only if residency sets are used

Remove custom cpu op from the M3 graph, express with stock ops (#26297) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS I

sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu ar

ggml-cuda: Allow transpose-free gemmv computation (#26171) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. Website: https://llama.app macOS/iOS: mac

tests : avoid building get-model.cpp many times (#26317) tests : remove get-model.cpp tests : fix quant type selection Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sil

llama-context : sync pending async copies before clearing embd_seq (#25676) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED mac

Test support for alternative conv layout (#25617) add bool cwhn = true to conv_2d test cases add layout check at graph building time extend layout checks for conv2d.cu kernel in CPU back-end kernel ne

vulkan: Support quantized concat (#25684) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Lin

server: support inp embd to generate next token (#26313) server: support embd for sampled token fix ~server_batch() Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silico

Learn how I modernized an old codebase of mine using stacked sessions and pull requests in the GitHub Copilot app. The post Stacked sessions and pull requests in the GitHub Copilot app appeared first

model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) (#25980) model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) Adds GLM-5.2 NextN/MTP as a --spec-type draft-m

add rdna3.5, and 3 to mmq configs so they can be tuned independently. (#26199) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED

RPC: add tensor_memset (#25912) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu

server : add trace logging for slot similarity checking (#26271) Adds trace logging in server-context.cpp for slot similarity checking during prompt cache slot selection, including skip reasons and si

vendor: update BoringSSL to 0.20260728.0 (#26241) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFrame

sycl: contiguous fast path + 32-bit index math for unary elementwise ops (#25946) sycl: contiguous fast path + 32-bit index math for unary elementwise ops sycl: use fastdiv for elementwise index math

ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory (#26141) ggml_cuda_should_use_mmq() selects MMQ purely from the quantization type. The current MMQ configurations are designed an

llama: move suppress_tokens handling to common/sampling (#26276) llama: move suppress_tokens handling to common/sampling address security issues rm has_logit_bias Website: https://llama.app macOS/iOS:

Disable -ffast-math on HIP (#25495) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ub

spec: add eagle3-v3 support for gpt-oss model (#25794) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XC

ggml-metal: FWHT kernel for metal backend (#25924) metal fwht wip shape guard and formatting formatting Formatting and typos Co-authored-by: YiChen Lv 63285796+forforever73@users.noreply.github.com fi

ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration (#22675) ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration cuda: added SSD CICD fixes for CUDA / HIP / MUSA / MSVC.

vulkan: add iq4_nl support back to FA (#24585) vulkan: add iq4_nl support back to FA I was originally concerned about wasting shared memory on the LUT, but it's small and unlikely to matter in practic

ggml : set output of view src (#25729) llama-graph: set_outputs to t->view_src change set_output to GGML_ASSERT about views not being outputs sampler : avoid views in outputs cont : fix dist sampler c

server: abstract llama_memory calls to common_memory (#26221) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)

opencl: skip the Adreno KQ/KQV image kernels for multi-stream batches (#26189) The Adreno KQ/KQV image1d kernels (ggml_cl_mul_mat_kq_kqv_adreno) ignore dim 3 entirely: the sub-buffer covers only nb02*

ggml-webgpu: Fix some binding alias issues to support all archs, fix recurrent-state-rollback test (#25931) Add overlap glu variant to support all archs, fix recurrent-state-rollback test format Fix a

model: Add Laguna-S-2.1 LLM_TYPE (#26233) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Lin

mtmd: Add Vision Support for Minimax-M3 (#25113) Add preliminary MiniMax-M3 support Text-only port that re-uses existing components: MiniMax-M2 style GQA with per-head QK-norm and partial rotary, Deep

server + ui: fix stream routes for model names containing a slash (#26137) server + ui: refactor resumable stream routes to query string conv_id The conversation id can embed a model name containing s

args: add -lm mlock where it mlocks but doesnt mmap (#26135) arg: add -lm mlock where it mlocks but doesnt mmap Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: rm unwanted docs changes Signed-off-by

ggml-cpu: Enable BF16 tiled gemm optimization on PowerPC (#26068) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (

common: fix explicit -md precedence over draft sidecar resolution (#26165) common: fix explicit -md precedence over draft sidecar resolution Follow-up of #25955, an explicit --model-draft file given w

tests : remove unnecessary sync in test-save-load-state (#26166) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x

ggml : adjust logic for offloading ops to weight's backend (#25832) ggml : adjust logic for offloading ops to weight's backend llama : dsv4 graph fixes Website: https://llama.app macOS/iOS: macOS Appl

sycl(build): parallelize ocloc invocations (#25903) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFra

fit : count nextn (MTP) blocks in n_gpu_layers so front layers stay on GPU (#26177) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISA

model: Add support for Nanbeige4.2 (#25994) support nanbeige4.2 model fix fix flake8 Lint check fix loop bound check and drop redundant head_dim Co-authored-by: root lizongqiang@kanzhun.com Website: h

common : add common_print_available_devices() (#26170) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,

mtmd: support MiMo-V2.5 audio input (RVQ-based model) (#26190) gguf converter for mimo audio fix conv cpp impl nits nits 2 Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple