Search: “apple”

cuda : fix CUB argsort corruption caused by in-place keys (#28389) argsort_f32_i32_cuda_cub called the one-shot DeviceRadixSort::SortPairs API with d_keys_in == d_keys_out (temp_keys, temp_keys). CUB'

metal : support qwen4exp hc ops (#29000) Add support for the new DSV4 HC op variants used by qwen4exp: hc_pre with per-element sigmoid gate (gated variant) hc_post with identity mixing (comb == nullpt

test-llama-archs : generate dummy test vocab (#29084) Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations

server : improve startup log messages (#29125) server-models : show source per model in log Show [source] tag (preset/models_dir/cache) per model instead of cryptic * marker Show HF hub cache path in

hexagon: enable support for TOP_K op (#29113) hexagon: enable support for TOP_K op hex-topk: thread single-row TOP_K, raise VTCM-based size cap hex-topk: fix TOP_K mdev row partitioning hex-topk: opti

vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 1024 experts (#28501) vulkan: raise the hoisted row-id limit for mul_mat_id to 512 experts The expert-count shader (count_experts.comp

cmake : fix build when GGML_CPU=OFF and GGML_CUDA=ON (#29026) fix: build fails when GGML_CPU=OFF and GGML_CUDA=ON fix: eol in examples/convert-llama2c-to-ggml/CMakeLists.txt file Website: https://llam

vocab : add ufakzeka pre-tokenizer (#29033) vocab : add ufakzeka pre-tokenizer vocab : move ufakzeka to the models list and regenerate the hash mapping Website: https://llama.app Attestations: https:/

vulkan: add IQ3_S MMQ matmul kernels (#28822) vulkan: add IQ3_S MMQ matmul kernels Make block_a_to_shmem do 2-byte loads (110 bytes is divisible by 2) Align the check, IQ3_S is also using K tile size

ggml-webgpu: fix supports_op condition for GET_ROWS (#28978) fix get_rows vec4 handling Add src strides checking to vec4_aligned of get_rows and the new test case. Website: https://llama.app Attestati

Model-Saver: Write the SWA pattern, 15 more architectures roundtrip (#29042) llama: read the SWA pattern as a period or a per-layer array Add llama_model_base::load_swa_pattern(), which reads sliding_

ggml : check for allocation failures to prevent crashes (#28149) ggml : check for allocation failures to prevent crashes wording Website: https://llama.app Attestations: https://github.com/ggml-org/ll

opencl: add bin kernel kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin (#28678) opencl: add A8 Q6_K non-MoE binary kernel opencl: fix layout compatibility Website: https://llama.app Attestations:

hexagon: HMX flash-attention head_dim padding (support DK=DV=72) (#26539) Allow HMX flash-attention to run with head_dim not a multiple of 64 (e.g. SigLIP head_dim=72), by operating on DK/DV rounded u

hexagon: im2col update (#29103) ggml-hexagon: accept 1D and padded IM2COL ops ggml-hexagon: make pure-DDR IM2COL kernel is_2D-aware ggml-hexagon: extend IM2COL DMA patch-embed fast path to 1D ggml-hex

opencl: add support for bin kernel flash_attn_f32_f16_bin (#29046) opencl: add flash_attn_f32_f16_bin opencl: guarded prefill fa Website: https://llama.app Attestations: https://github.com/ggml-org/ll

arXiv:2609.20747v1 Announce Type: cross Abstract: Reinforcement learning constitutes a promising approach owing to its potential for superhuman performance and self-learned policies. However, its appl

[SYCL] Fix function signature for ggml_backend_sycl_split_buffer_type (#28981) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48082811 macOS/iOS: macOS App

vulkan: support qwen4exp hc ops (#28988) vulkan: support qwen4exp hc ops fix stale comment [no-ci] Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48109146

vulkan: fix buffer_reference alignment in im2col shaders (#28996) Both im2col.comp and im2col_3d.comp declare D_ptr without an explicit buffer_reference_align, so glslang emits writes through it as Al

opencl: fix various warnings (#28984) opencl: fix warnings opencl: fix warnings for non adreno Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48132480 macO

gguf : align the data section relative to the GGUF start, not the file (#28993) gguf : align the data section relative to the GGUF start, not the file gguf_init_from_file_ptr reads a GGUF from the cur

chat : add message delimiters to the DeepSeek V3.2/V4 parser (#29008) chat : add message delimiters to the DeepSeek V3.2/V4 parser Assisted-by: Claude Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaere

vulkan: split buffers and debug code into separate files, add shared headers (#28732) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48170206 macOS/iOS: ma

openvino : Update OpenVINO to 2026.4;fix clangd,MSVC warnings; (#29009) Update to openvino-2026.4 Update OV docs ggml-openvino : fix clangd and MSVC warnings fix int to ptr cast, more internal linkage

model : extend Nemotron MTP support (#29018) first fix removed unnecessary declarations Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48188464 macOS/iOS:

model : skip gate_up_exps if TENSOR_SKIP is set (#29014) required for qwen35moe if MTP tensors are fused but not loaded Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/a

Hey all, Henry from Cactus Compute here, I kinda wanted to share our latest model and get feedback from the family :) Needle 3 is a small foundation model for automation: you give it the functions you

ci : add self-hosted webgpu to hf-jobs (#28712) add self-hosted vulkan and webgpu to hf-jobs try t4-medium cont : adjust cpu backend threads try t4-small again restore cm jobs Co-authored-by: Georgi G

vulkan: make MUL_MAT_ID BN/2 tail unconditional (#28923) Use BN/2 as the default for BNover2 and as the disabled fallback for BNover4, and remove the enable gate from the MUL_MAT_ID BN/2 branch. The B

HIP: broaden MoE ncols_opt tile heuristic on RDNA3.5 architecture (#28935) It's found the MoE ncols_opt tile heuristic needs to be broadened to include the RDNA3.5 architecture. The code change is imp

rpc : invalidate cached compute graph when a referenced buffer is freed (#24292) The server caches the most recent compute graph per device so that GRAPH_RECOMPUTE can re-execute it without resending

spacemit : fix wrong transpose function for int16 data (#25161) The sizeof(int16_t) branch in permute_transpose_impl calls rvv_transposed_s32_mn_to_nm instead of rvv_transposed_s16_mn_to_nm. This is a

model : add support for HrmTextForCausalLM (DFM Mimir 1B) (#27625) model : add support for HrmTextForCausalLM (DFM Mimir 1B) HRM-Text runs two transformer stacks (low, high) in an alternating cycle ov

hexagon: accept the zeroed rope probe in supports_op (#28995) llama probes weight placement with a rope where all params are 0, so rejecting n_dims == 0 or freq_base == 0 puts rope_freqs on the CPU. T

hexagon: Support for K-Quants Q4_K and Q6_K (#28994) implement q6k/q4k kernels Squashed from: feat: implement q6k kernel hex-q6k: improve unpack accuracy hex-q4_k: add support for Q4_K kernels Co-auth

Enable CUDA graph for MTP draft (#28549) Improve CUDA graph usage for MTP Rename field Address review feedback Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestatio

TP: fix split state and granularity for fused QKV gemma4, qwen35 (#28965) model: calculate split states for attn_qkv from n_head * n_embd_head_k required for gemma4 with --fuse-qkv, where n_embd is 53

Confidential Computing keeps your data from being tracked or used in three ways: - End-to-end encrypted inference - Hardware-enforced isolation that extends to the GPU with @nvidia Confidential Comput

arXiv:2609.13846v1 Announce Type: cross Abstract: Delayed Tensor Parallelism (DTP) removes the blocking all-reduce of tensor-parallel Transformer inference. Every device adds its own partial output to

metal : add FA kernels for HSK=96, HSV=64 (MiniCPM3) (#28599) metal : add FA kernels for HSK=96, HSV=64 (MiniCPM3) MiniCPM3 sets attention.key_length to 96 and does not set attention.value_length, whi

opencl: add generic ssm_scan (#28881) opencl: add generic ssm_scan opencl: fix whitespace Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47601188 macOS/iOS

OpenVINO: optimize stateful decode and GPU MoE inference (#28638) exclude GPU/NPU failing POOL_2D case Fix pool case ggml-openvino: fix stateful decode for Gemma-4 per-layer-type head sizes ggml-openv

vulkan: support sparse Flash Attention (#28105) vulkan: add sparse Flash Attention support for DSV4/GLM tune implementation add tests avoid nondeterministic atomicAdd add cm2 decode vector support sim

models : move build_arch_graph() after graph() template specialization (#28934) Move build_arch_graph()'s function definitions after the graph and graph template specializations have been explicitly d

cuda: support row-contiguous SUM_ROWS (#26308) cuda: support row-contiguous SUM_ROWS organize the code and add GGML_OP_MEAN to support row-contiguous tensors using the same shared kernel, and add a te