b10753
hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (#28217) hexagon: fix bug ne[2] printed in proc_op_req prep-src log hexagon: add shape/VTCM farf logs to cpy, get/set rows, gdn Website
Knowledge catalogue
hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (#28217) hexagon: fix bug ne[2] printed in proc_op_req prep-src log hexagon: add shape/VTCM farf logs to cpy, get/set rows, gdn Website
opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632) opencl: clamp the q4_K decode GEMV's fetch row on a padded x-grid opencl: enforce the tiling contract of the image KQ/KQV GEMMs open
vulkan : only request VK_KHR_shader_bfloat16 extension if supported (#28155) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44637403 macOS/iOS: macOS Apple
vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (#27449) vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec when NUM_COLS > 4. 5x perf at n=8 Assisted-by: Claude
hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (#28202) hex-mm: fuse QKV and FFN matmuls that land on HMX hex-mm: remove hardcoded ne[1] < 32K restriction hex-get-rows: explicitly reject repacked Q8
ggml: avoid KleidiAI buffer type init on dispatch (#27891) Co-authored-by: Acmmi acmmi@Acmmis-MacBook-Air.local Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestati
mtmd: Fix Qwen3-tts-0.6b (#28231) mtmd: load the qwen3-tts code predictor proj_in as optional The talker and the code predictor share the hidden size on the 0.6B checkpoints, so the reference builds n
mtmd: support DeepSeek-V4-Flash-Vision-Exp (#28133) mtmd: support DeepSeek-V4-Flash-Vision-Exp handle min/max token counts from CLI rm debugging use GGML_ROPE_TYPE_VISION nits apply review comments co
common, server : enable preserve_reasoning kwarg by default, log its effective state (#28174) common, server : enable preserve_reasoning kwarg by default, log its effective state If the preserve_reaso
ggml-cuda : remove unused vars (#28235) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44811308 macOS/iO
model: correctly support input vision for deepseek4 (#28154) model: correctly support input vision for deepseek4 nits Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/att
Update ROCm to 10.0.0 release (#27803) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44848715 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (
metal : fix memory query under low-memory conditions (#27701) metal: Fix memory query under low-memory conditions Simply variable name Co-authored-by: Georgi Gerganov ggerganov@gmail.com Write it even
From loop engineering to harnesses, squads, and open weights, the GitHub Podcast breaks down the AI terms showing up in developer conversations. The post Decoding the new AI lingo: Loops, harnesses, s
Why shorter outputs can cost more, and how GitHub Copilot reduces wasted work across the complete coding task. The post How we make AI coding more cost efficient without sacrificing task quality appea
Ollama released version v0.33.3 on September 2, 2024, based on commit 5ec5804 and signed with GitHub’s verified signature (GPG key B5690EEEBB952194). Key changes include default parameter definitions
qwen4exp: sum the indexer heads by slices (#28023) qwen4exp: sum the indexer heads by slices The head reduction went through a transpose and a sum_rows over ne[1], which left sum_rows with ne0 = 4, on
qwen4exp: support recurrent state rollback (#28123) MTP speculative decoding needs the target state to move back by the number of rejected draft tokens. Without rollback support the context is classif
ci: Bump ggml-org/ccache-action to v1.2.24 (#28083) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44375174 macOS/iOS: macOS Apple Silicon (arm64) macOS Ap
metal: enable Metal 4.0 tensor API on M5+/A19+ (#27461) metal : request Metal 4.0 language version for the tensor API metal : load the tensor API kernels from a separate metallib tests : add external-
tests : fix log verbosity for test-llama-archs (#28147) tests : fix log verbosity for test-llama-archs cont : naming cont : add note Website: https://llama.app Attestations: https://github.com/ggml-or
qwen4exp: fix seq_cp, block position keying, mtmd input, cuda abort, add tests (#27941) qwen4exp: follow up fixes -kvu NaN collapse fix Assisted-by: Claude indexer cache ext.x/ext.y restore fix Assist
sycl : support limit max alloc memory within 2GB for host-pinned memory (#27559) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44438607 macOS/iOS: macOS A
metal : add fa-vec tuning for M2 Max (#28015) Rows for M2 Max (30 GPU cores) collected with 'ggml-metal-tuning fa-vec --dtype f16,q8_0', pasted into fa_vec_tuned_table. ref: https://github.com/ggml-or
metal : fix more leaks due to missing autoreleasepools (#27883) metal : fix more leaks due to missing autoreleasepools metal : rename variable metal : fix another missing pool warning Co-authored-by:
model : load hparams.n_layer_nextn before n_layer() calls (#28159) load hparams.n_layer_nextn before n_layer() calls remove duplicate loads Website: https://llama.app Attestations: https://github.com/
metal : add fa-vec tunings for A18 Pro (MacBook Neo) (#28152) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44474754 macOS/iOS: macOS Apple Silicon (arm64
metal : add fa-vec tuning for M2 Pro (#28122) metal: add fa-vec tuning for M2 Pro metal : update fa-vec tuning for M2 Pro with new dtypes Website: https://llama.app Attestations: https://github.com/gg
context : autoscale n_ctx_train when yarn scaling specified (#28030) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44499127 macOS/iOS: macOS Apple Silicon
kv-cells: look up the n-gram history in the sequence position index (#28040) get_prev_tokens() rebuilt a (seq, pos) -> token hash map on every ubatch by walking all used cells, while llama_kv_cells al
cuda: fuse MoE weighted expert reduction (#25952) cuda : fuse MoE weighted reduction (mul + view + add) The MoE combine tail currently writes weighted expert outputs to global memory before reducing t
ggml: add SWIGLU_CLAMP (#27930) ggml: add SWIGLU_CLAMP add vulkan shader Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44034152 macOS/iOS: macOS Apple Sil
kv-cells: stop the sequence scan once all sequences are seen (#28011) for_each_token_in tested all LLAMA_MAX_SEQ sequences for every used cell, while a cell almost always belongs to one. The scan now
ggml : fix ggml_backend_buft_get_alloc_size() guard (#28038) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44038725 macOS/iOS: macOS Apple Silicon (arm64)
rpc: avoid serializing buffers from other servers (#26500) rpc: avoid serializing buffers from other servers Only include remote buffer pointers when the buffer belongs to the RPC dispatcher receiving
metal : add remaining Q4_1/Q5_0/Q5_1 fa-vec tunings for M2 (#28017) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44044108 macOS/iOS: macOS Apple Silicon
hexagon: fix CPY fence bug (#28033) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44046740 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm
vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (#28032) vulkan: add top-k radix sort shader for k >= 1024 add Qwen 3.8 Flash Next top-k tests add top-k qsa fusion clean up code Websi
ggml : add MUL_MAT to the list of ops that may need additional memory (for WebGPU) (#28071) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://githu
vulkan: tune mat-vec rows for batched inference on Strix Halo (#27909) vulkan: RDNA3 static mat-vec rows above four columns On RDNA3 above four columns a static 4 rows for all types benches faster tha
spec : fuse the DFlash encoder into the KV cache injection (#27310) dflash : fuse the encoder into the KV injection decode The encoder is a single fc + norm, but running it as a separate llama_encode
sycl : Enhance to get the free memory of Intel GPU (#27968) enhance get mem info by l0 an SYCL API remove debug code, format the code update SYCL.md for GGML_SYCL_GET_MEM_API Website: https://llama.ap
CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (#27621) CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion
metal : add fa-vec tunings for M1 (#28078) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44172949 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
ROCm: add radix TOP_K for long rows (#27466) ROCm: add radix TOP_K for long rows Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44178618 macOS/iOS: macOS A
webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_get() implementation (#28045) webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_ge
opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance (#26438) opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse opencl: Q4_K mul_mm 8x8 tile fot In
kv-cache : optimize restoring non-contiguous cells (#27991) kv cache : batch state restore scatter reads per contiguous run When restoring state into non-contiguous destination cells (e.g. a prompt-ca
AVX2: Speed up large batch size prompt processing of IQ models (#27402) Batched gemm for grid IQ quants Style updates and a bit more performance Clean up comments Move code around Vectorize IQ panel d
metal : add concat support for quantized types (#28116) Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/442
CUDA: XOR swizzle flash attn K,V smem fp16 tiles (#25635) CUDA: XOR swizzle flash attn K,V smem fp16 tiles Signed-off-by: ynankani ynankani@nvidia.com Fix use 64bit generic pointer instead of 32bit sh
metal : add fa-vec tunings for M1 Ultra (#28088) metal : add fa-vec tunings for M1 Ultra metal : move M1 Ultra tunings after M1 Max section metal : remove duplicate blank line Website: https://llama.a
metal : add fa-vec tunings for M2 (#27940) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43902029 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
ggml: allow passing alloc dependencies in graph_optimize (#27301) ggml: allow passing alloc dependencies in graph_optimize add alloc dep tests add TODO about using flat array Website: https://llama.ap
memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.c
metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (#25648) metal : fail closed on mul_mat shapes with missing F16 kernels metal : abort on nil pipeline in encoder_set_pipeline metal : ad
sycl: split long rows in TOP_K instead of one work-group per row (#27847) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43958406 macOS/iOS: macOS Apple Si
hexagon: support for device discovery and create sessions on demand (#27785) hex-devices: add support for lazy session allocation and cleanup dev interfaces Co-authored-by: Marco Colombo mcolombo@qti.
rpc : fix pre-rdma macOS versions (#27815) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43962220 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
vendor : update cpp-httplib to 0.54.0 (#27919) vendor : update cpp-httplib to 0.54.0 vendor : update cpp-httplib to 0.54.0 and 0.54.1 Website: https://llama.app Attestations: https://github.com/ggml-o