b10174
model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) (#25980) model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) Adds GLM-5.2 NextN/MTP as a --spec-type draft-m
Knowledge catalogue
model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) (#25980) model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) Adds GLM-5.2 NextN/MTP as a --spec-type draft-m
add rdna3.5, and 3 to mmq configs so they can be tuned independently. (#26199) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
RPC: add tensor_memset (#25912) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu
server : add trace logging for slot similarity checking (#26271) Adds trace logging in server-context.cpp for slot similarity checking during prompt cache slot selection, including skip reasons and si
vendor: update BoringSSL to 0.20260728.0 (#26241) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFrame
sycl: contiguous fast path + 32-bit index math for unary elementwise ops (#25946) sycl: contiguous fast path + 32-bit index math for unary elementwise ops sycl: use fastdiv for elementwise index math
ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory (#26141) ggml_cuda_should_use_mmq() selects MMQ purely from the quantization type. The current MMQ configurations are designed an
llama: move suppress_tokens handling to common/sampling (#26276) llama: move suppress_tokens handling to common/sampling address security issues rm has_logit_bias Website: https://llama.app macOS/iOS:
Disable -ffast-math on HIP (#25495) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ub
spec: add eagle3-v3 support for gpt-oss model (#25794) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XC
ggml-metal: FWHT kernel for metal backend (#25924) metal fwht wip shape guard and formatting formatting Formatting and typos Co-authored-by: YiChen Lv 63285796+forforever73@users.noreply.github.com fi
ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration (#22675) ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration cuda: added SSD CICD fixes for CUDA / HIP / MUSA / MSVC.
vulkan: add iq4_nl support back to FA (#24585) vulkan: add iq4_nl support back to FA I was originally concerned about wasting shared memory on the LUT, but it's small and unlikely to matter in practic
ggml : set output of view src (#25729) llama-graph: set_outputs to t->view_src change set_output to GGML_ASSERT about views not being outputs sampler : avoid views in outputs cont : fix dist sampler c
server: abstract llama_memory calls to common_memory (#26221) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)
opencl: skip the Adreno KQ/KQV image kernels for multi-stream batches (#26189) The Adreno KQ/KQV image1d kernels (ggml_cl_mul_mat_kq_kqv_adreno) ignore dim 3 entirely: the sub-buffer covers only nb02*
ggml-webgpu: Fix some binding alias issues to support all archs, fix recurrent-state-rollback test (#25931) Add overlap glu variant to support all archs, fix recurrent-state-rollback test format Fix a
model: Add Laguna-S-2.1 LLM_TYPE (#26233) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Lin
Generative AI can make cloud costs difficult to predict. A single five-word prompt can run complex operations and generate significant costs. Traditional metrics like requests per second no longer hel
mtmd: Add Vision Support for Minimax-M3 (#25113) Add preliminary MiniMax-M3 support Text-only port that re-uses existing components: MiniMax-M2 style GQA with per-head QK-norm and partial rotary, Deep
args: add -lm mlock where it mlocks but doesnt mmap (#26135) arg: add -lm mlock where it mlocks but doesnt mmap Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: rm unwanted docs changes Signed-off-by
ggml-cpu: Enable BF16 tiled gemm optimization on PowerPC (#26068) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (
common: fix explicit -md precedence over draft sidecar resolution (#26165) common: fix explicit -md precedence over draft sidecar resolution Follow-up of #25955, an explicit --model-draft file given w
tests : remove unnecessary sync in test-save-load-state (#26166) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x
ggml : adjust logic for offloading ops to weight's backend (#25832) ggml : adjust logic for offloading ops to weight's backend llama : dsv4 graph fixes Website: https://llama.app macOS/iOS: macOS Appl
sycl(build): parallelize ocloc invocations (#25903) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFra
fit : count nextn (MTP) blocks in n_gpu_layers so front layers stay on GPU (#26177) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISA
model: Add support for Nanbeige4.2 (#25994) support nanbeige4.2 model fix fix flake8 Lint check fix loop bound check and drop redundant head_dim Co-authored-by: root lizongqiang@kanzhun.com Website: h
common : add common_print_available_devices() (#26170) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,
mtmd: support MiMo-V2.5 audio input (RVQ-based model) (#26190) gguf converter for mimo audio fix conv cpp impl nits nits 2 Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
mtmd: fix android build (#26150) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubunt
metal : add f16 type support to leaky relu (#25981) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFra
args: refactor mlock/mmap/directio into load-mode (#20834) args: overhaul mmap/mlock/dio into single arg Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: update docs with llama-gen-docs Signed-off-by
CUDA: fix external compilation of q1_0 MMQ (#25778) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFra
hexagon: fix Windows crash when op_poll is enabled (#26029) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) i
ggml: enable PowerPC backend variants on AIX (#25983) ggml: enable PowerPC backend variants on AIX Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMak
Fix DeepSeek4 crafted template (#25414) chat: fix DS4 template to explicitly follow reference behavior Support DeepSeekv4 flag (drop_reasoning). fix: hook DS3.2 parser for DS4 as well fix: add tool re
common: infer the speculative type from the draft repo sidecars (#25989) With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars and no --spec-type given, the draft resolved to a full mo
hexagon: activation ops update (#25974) hex-geglu: optimized all-in-one geglu microkernel hex-geglu: enable non-contiguous src and strided DMA hex-act: enable non-contiguous srs and strided DMA for re
CUDA: Improve NVFP4 W4A4 activation quantization (#25730) Squash history before conflict-resolution during rebase on master WIP commit Add 32-byte loads, restore per-block amax Use nvfp4x4 intrinsic w
arXiv:2607.20237v1 Announce Type: new Abstract: Rehabilitation scoring systems are most useful when their outputs can be reviewed and interpreted within clinical workflows. This study presents PhaseAw
cuda: add sqrt_softplus in topk-moe for dsv4 (#25896) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCF
hexagon: check tensor type when reusing descriptors (#25968) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)
mtmd : use align_corners for qwen3vl vision position embedding interpolation (#25781) The Qwen3-VL learned position embedding is interpolated to the runtime patch grid with the default bilinear+antial
Add support for Laguna XS.2 & M.1 (#25165) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Li
llama-arch: fix DeepSeek4 APE tensor op (#25945) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramew
cuda: GET_ROWS quants (#25962) cuda: add k-quant support to GET_ROWS Device-side embedding lookups require GET_ROWS to handle the k-quants used by common GGUF recipes (Q4_K_M stores token_embd as q6_K
webgpu : add CONV_2D_DW (depthwise conv2d) kernel (#25847) webgpu : add CONV_2D_DW (depthwise conv2d) kernel Implement GGML_OP_CONV_2D_DW for the WebGPU backend, ported from the Vulkan backend's conv2
ci : fix SYCL package shared library lookup (#25987) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFr
🎨 Meet Qwen-Image-3.0 — the third generation of our foundational image generation model. If 1.0 was about 'Precision,' and 2.0 added 'Variety, Completeness, Beauty & Authenticity,' then 3.0 comes down
Earlier this month I hosted a fireside chat session at the AI Engineer World's Fair with Cat Wu and Thariq Shihipar from Anthropic's Claude Code team. We talked about Claude Code, Claude Tag, Fable, c
opencl: disable FA and MoE weights repack to work around compiler issues for Adreno 850 GPU (#25745) opencl: workaround for A850 compiler compat opencl: fix DX compiler version parsing and cleanup Co-
ci : add official website link to release notes (#25728) Assisted-by: pi:llama.cpp/Qwen3.6-27B Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI en
[SYCL] Flash Attention with XMX engine via oneDNN (#25222) [SYCL] F16 (default) Flash Attention with XMX engine via oneDNN graph API; Qwen3.6-27b-Q8_0 prefill speed up x1.21 at p=512 and x4.26 at p=80
sycl: Increase minimum buffer size for USM system allocations (#25525) Raise the threshold for minimum buffer size from 1 GiB to 4 GiB, based on real-world experiments of overcommitting device memory
DeepseekV4: reduce graph splits (#25702) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu
server : refactor prompt cache state ownership (#25649) server : clear checkpoints upon prompt clear server : move the prompt state data to the server_prompt_cache Assisted-by: pi:llama.cpp/Qwen3.6-27
arXiv:2606.30294v1 Announce Type: new Abstract: Live product demonstrations are a recurring, high-cost activity in software organizations: a human presenter must select features, dispatch the correspo
arXiv:2606.11079v1 Announce Type: new Abstract: Evaluation remains a critical bottleneck for interactive agent development. Existing evaluation methods often rely on static benchmarks, which fail to c
arXiv:2606.03203v1 Announce Type: new Abstract: Computer-use agents could automate repetitive screen-based clinical work, but their reliability in medical graphical user interfaces remains largely unv