Knowledge catalogue
Hold right Ctrl, speak, release. faster-whisper (large-v3-turbo) transcribes on your own GPU, then qwen2.5:3b-instruct through Ollama cleans the text up: filler words out, punctuation in, spoken self-
Pathway's Baby Dragon Hatchling (BDH) is a brain-inspired, post-transformer architecture that reasons in latent space instead of emitting chain-of-thought tokens. See how Pathway develops and scales B
vulkan: add TQ1_0 support (mm, mat-vec, mat-vec-id, dequant, get_rows) (#27765) vulkan: add TQ1_0 support (mm, mat-vec, dequant, get_rows) vulkan: pack TQ1_0 powers of 3 into a 32-bit constant Replace
vulkan: rms_norm fusion opportunities (#28024) Support RMS_NORM + MUL + ADD (+ MUL) and RMS_NORM + VIEW + SET_ROWS. Extend ROPE + VIEW + SET_ROWS to support IMROPE. Worth around 4% in gemma4 on my sys
ggml: allow backend inputs to not create another split (#28387) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45674860 macOS/iOS: macOS Apple Silicon (arm
ggml-cuda: fix divergent barrier in f16 flash attention (#27870) ggml-cuda: fix divergent barrier in f16 flash attention ggml-cuda: avoid duplicate metadata pointer setup Website: https://llama.app At
caps : recheck typed content if template checks for string (#28511) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45690927 macOS/iOS: macOS Apple Silicon
vulkan: support type-aligned GET_ROWS (#28253) vulkan: fall back to CPU for GET_ROWS with misaligned offsets The Vulkan GET_ROWS shader asserts when a tensor's backing-buffer offset plus view_offs is
CUDA: branchless Q4_K/Q5_K unpack to speed up mmvq, L2 prefetch on DGX Spark (#26705) Update Q4_K and Q5_K to use branchless computation, which stops the scale unpack being re-executed for every colum
ggml: add gfx90c HIP support (#26454) ggml: add gfx90c HIP support ggml: make gfx90c HIP support compliant with specifications Website: https://llama.app Attestations: https://github.com/ggml-org/llam
vulkan: add DeepSeek-V4 hyper-connection fused ops (DSV4_HC_COMB/PRE/POST) (#26578) vulkan: add DeepSeek-V4 hyper-connection fused ops (DSV4_HC_COMB/PRE/POST) CUDA has these ops from the DeepSeek-V4 m
tests : initialize the L2_NORM batch array (#28553) tests: bind the L2_NORM batch count to a local GCC cannot prove the loop fills norms up to the index read after it while the bound is a class member
metal : add remaining fa-vec tunings for M2 Max (#28458) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45551940 macOS/iOS: macOS Apple Silicon (arm64) mac
common: add --log-jsonl (#28437) common: add --log-jsonl rename unknown to none Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45560949 macOS/iOS: macOS Ap
grammar : fix max repetition threshold (#28469) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45568343 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
cuda: fixes races in mmid and mmf (#28475) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45586039 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
opencl: properly choose weights pack for q4_K, q5_K mul_mat (#28402) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45608946 macOS/iOS: macOS Apple Silicon
[Model] Support for Spark2_5ForCausalLM implementation (#27868) Add Spark3 Model rename spark3 -> spark2_5 Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: dongjiang d
models : fix GDN normalization from max to rsqrt (#28068) models: use flash-linear-attention's l2norm for gated delta net q/k The GDN q/k normalization is defined by flash-linear-attention as l2norm(x
convert : add --fuse-qkv flag to fuse Q/K/V into QKV during HF-to-GGUF conversion (#22780) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45621954 macOS/iO
An agent for every citizen Engineers to upgrade our institutions Robots as a service for all $1 valuation for local individuals & institutions Market rates for everyone else Owns the last mile of inte
sycl: attribute device allocations by site (GGML_SYCL_MEMTRACE) (#27631) define two new environment variables to better understand how much memory is being allocated, and when. This has been invaluabl
sycl : fix test-backend-ops CI break && restore Kronecker product FWHT support (#28016) (#28254) Reapply 'sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 12…' (#28184) This reverts
metal : fix memory leak in early return (#28399) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45438612 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
SYCL: Refactor GGML_SYCL_ENABLE_MKL_FA to global var (#26863) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45153754 macOS/iOS: macOS Apple Silicon (arm64
sycl: fuse rms_norm+mul+add and add+add residual chains (#27610) Fuse RMS_NORM+MUL+ADD and ADD+ADD under GGML_SYCL_ENABLE_FUSION. ADD+ADD uses the same binbcast indexing and type matrix as standalone
src : add n_expert_used_max function (#28323) src : add n_expert_used_max function With Commit c61b98b ('model: add NVIDIA Nemotron-3-Puzzle-75B-A9B (NemotronHPuzzle) support (#25444)') it is now poss
ggml-cpu(s390x) : fix q5_1 uninitialized v_acc (#28332) Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45199911
common : make build info output stream configurable (#28322) Let llama_print_build_info write to a caller-provided FILE* instead of hardcoding stderr. The parameter defaults to stderr so existing call
llama.cpp : bump version to 0.4.0 (#28386) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45314398 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
opencl: add Adreno xmem SDPA path (#26331) opencl: add Adreno xmem SDPA path Assisted-by: Codex Removed the Adreno-specific queue profiling override Clean up formatting 修复数值误差优化gqa/mask attn Assisted-
opencl: extend the elementwise and data‐movement op coverage (#27633) opencl: add extended elementwise unary ops (sgn, step, elu, hardswish, hardsigmoid, floor, ceil, round, trunc) Adds nine GGML_UNAR
metal : add remaining fa-vec tunings for M3 (#28396) addition of m3 in fa_vec_tuned_table adding q4_0,q4_1,q5_0,q5_1 in ggml-metal-tuning Fix formatting in ggml-metal-tuning.cpp Website: https://llama
metal : add fa-vec tunings for M3 (#28236) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44857719 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic
mtmd: add mtmd_tokenize_from_parts() (#28250) add mtmd_tokenize_from_parts use it in mtmd-cli move add_special to call level Website: https://llama.app Attestations: https://github.com/ggml-org/llama.
ggml-hexagon: add F16 support for unary ops (#28228) Extend the HTP backend's F16 unary op coverage to include ABS on top of the existing NORM/RMS_NORM/L2_NORM/SCALE/CLAMP/SQR/SQRT set. Add hvx_abs_f1
server : accept data: URLs for input_video and input_audio (#27735) server : accept data: URLs for input_video and input_audio input_video and input_audio passed accept_base64_uri=false to handle_medi
finetune: fix no KV cache (#27199) training: fix no KV cache apply @ ggerganov suggestion Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44871971 macOS/iOS
mtmd: fix idefics3 preproc (#28273) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44875193 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm
model: add NVIDIA Nemotron-3-Puzzle-75B-A9B (NemotronHPuzzle) support (#25444) hparams: add per-layer n_ff_exp/n_expert_used arrays with scalar-or-array loading G1/G2 infrastructure for variable-per-l
sycl: reduce redundant work in Q4_K multi-column MMVQ (#27062) sycl: Q4_K Weight unpack optimization and reuse between destination Columns sycl: Q4_K small N (N=2..4) + two output rows by subgroup reu
misc : prevent RAM peaking at model loading stage (#27483) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44939788 macOS/iOS: macOS Apple Silicon (arm64) m
sycl : enhance the api to support peer-to-peer copy (#27550) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44948405 macOS/iOS: macOS Apple Silicon (arm64)
vulkan: fix FA dequant path engagement (#28190) Skip the nb[3] check when ne[3] == 1, the shader never reads it for a single stream. Cache views carry the full-buffer stride there, so the old check re
CUDA: Allow concurrent streams per split for multi-GPU (#28198) CUDA: Allow CUDA optimization per split for multi-GPU. Previous guard caused multi-GPU to skip the graph optimization. The graph is alre
mtmd : add const in various places (#28307) mtmd : mark context as const in more methods Mark mtmd_context as const in: mtmd_bitmap_init_lazy mtmd_tokenize mtmd_tokenize_from_parts mtmd_helper_support
metal : fix glu dispatch with ne00 = 1 (#28306) metal : fix glu dispatch with ne00 = 1 tests : disable ill-defined tests Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/
metal : add sparse FA (#28098) metal : support n_kv_max sparse mask hint in flash attention vec kernel add kernel_flash_attn_ext_vec_idx: compacts finite mask entries into a per-row index list (Hillis
mtmd: propagate const to preproc class (#28310) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44990064 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple
tune MMVQ to MMQ crossover for SM87 (#28285) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45087167 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sil
opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations (speculative decoding/MTP) (#26477) opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations opencl: guard q4_K/q
common/json-schema : fix GBNF grammar generation for empty object schemas (#28279) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45102223 macOS/iOS: macOS
llama: fix whole source code rebuilt on each new commit (#28278) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/45114012 macOS/iOS: macOS Apple Silicon (ar
Two weeks, one rule, and hundreds of entries from nearly 50 countries. Here's who took the four titles, and a look at everyone who made the final ten! Entries opened August 20 and closed September 1.
🏆 #1 overall on Code Arena, and top of the Pareto frontier at 5/MToken. Thanks! @arena Give Qwen3.8-Max-0902 a spin on QwenCloud.🥳 Qwen3.8-Max-0902 by @Alibaba_Qwen just debuted at #1 overall in Code
metal : add metallib build support for xcframework (#28163) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44563342 macOS/iOS: macOS Apple Silicon (arm64)
hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (#28217) hexagon: fix bug ne[2] printed in proc_op_req prep-src log hexagon: add shape/VTCM farf logs to cpy, get/set rows, gdn Website
opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632) opencl: clamp the q4_K decode GEMV's fetch row on a padded x-grid opencl: enforce the tiling contract of the image KQ/KQV GEMMs open
vulkan : only request VK_KHR_shader_bfloat16 extension if supported (#28155) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44637403 macOS/iOS: macOS Apple
vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (#27449) vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec when NUM_COLS > 4. 5x perf at n=8 Assisted-by: Claude