Search: “apple”

rpc: include nb in the get_alloc_size cache key and floor the result at ggml_nbytes (#29283) rpc : include nb in the get_alloc_size cache key and floor the result at ggml_nbytes cont : remove redundan

HIP: bump HIP_VERSION requried for fp8 to avoid missing __hip_fp8_e4m3 support in 6.2 (#29231) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50216799 macO

llama : add llama_prec_policy + model-driven W4A4 path (#24364) Rebase and update based on #26675 Signed-off-by: ynankani ynankani@nvidia.com CI failure fix(launh_bounds overload on HIP) and cleanup S

metal : split fa kernels into per-dtype libraries (#29329) metal : split fa kernels into per-dtype libraries Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp cont : minor fix comment Website: ht

metal: FWHT kernels for block widths above 512 (#29095) metal: FWHT kernels for block widths above 512 The Metal FWHT covers widths 64 to 512, one row per simdgroup with N/32 values per lane. Wider bl

common : extract shared unicode path/string helpers (#29415) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestati

Fixing the vulkan build issue of legacy GLSLC version that has no cooperativeMatrix API support (#29373) (#29409) vulkan : fix build issue of legacy glslc version by adding GGML_VULKAN_COOPMAT_GLSLC_S

opencl: add bin kernel kernel_gemm_noshuffle_q5_k_f32_32b_trans_ila_a8_bin, kernel_gemm_noshuffle_q5_k_q8_1_dp4a_ila_a8_bin (#29401) opencl: add A8 Q5_K non-MoE non dp4a + dp4a binary kernel opencl: f

mtmd: fix mel preprocessor in LFM2 audio (#29403) which resulted in different greedy transcripts for 4.5% of English and 6.5% of Japanese test utterances. In Japanese, some differences changed entire

common,rpc : simplify fs_create_directory_with_parents() (#29432) The original function was broken on Windows for some unicode paths Paths without a trailing separator now create the last directory to

For those who have V100 cards, I wanted to point you to 1Cat-vLLM, a vLLM fork that enables optimized serving for these cards. Showing stats for Qwen3.6-35b comparing a Strix Halo with a hughly optimi

FYI the engine's default kernel rules were measured on smaller chips (16/20-core M5s and a 32-core M4 Max), so a 40-core M5 Max runs guesses. Splash's repo includes a developer tool, “make tune-kernel

test-save-load-state : print a per-model results table in --models mode (#29316) test-save-load-state : print a per-model results table in --models mode in --models mode the output was very heavy: eve

server,common : fix the GCC 12 stringop-overread false positive (again) (#29325) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org

model : add Ling 3.0 VL support (#29151) model : fold Ling 3.0 VL into the BailingMoeV3 architecture Assisted-by: Scout model : keep shared NORM rope list intact when gating bailingmoe3 on mrope secti

cuda : add conv3d with implicit GEMM (#29137) cuda : add conv3d with implicit GEMM cuda : refine conv3d implicit GEMM and handle empty kernels Website: https://llama.app Attestations: https://github.c

vulkan: tune KHR cooperative matrix support for Adreno GPUs (#29328) Enable coopmat support for Vulkan backend Fixed the mul_mat_s Removed the debug statement Website: https://llama.app Attestations:

vulkan: handle misalignment in conv_2d and conv_3d (#29365) vulkan: handle misalignment in conv_2d and conv_3d fix test-backend-ops print Website: https://llama.app Attestations: https://github.com/gg

vulkan: int8 coopmat1 matmul implementation for AMD RDNA3 and RDNA4 (#27952) vulkan: add int8 coopmat quantized matmul shader apply scales inline use scalar sums probe and directly access coopmat valu

llama: add llama_batch_ext (#24669) (wip) add llama_batch_ext wip updated design updated impl change signature unused var demo common_prompt_batch_decode fix pos tmp disable test-batch-alloc fix compa

hexagon: dynamic quantizer improvements (#29395) hexagon: fix accuracy issue in Q8_0 N=1 MUL_MAT hex-quant: fix register spills hex-mm: use dma for all dyn.quant paths Co-authored-by: Aparna M P aparm

sync : ggml (#29396) ggml : bump version to 0.25.2 (ggml/1642) ggml : fix ubsan error in ggml_graph_nbytes (ggml/1644) ggml : bump version to 0.25.3 (ggml/1645) sync : ggml Website: https://llama.app

metal : optimize sparse FA + clean-up (#29377) metal : cache sparse FA indices in shared memory Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp metal : simplify shared memory size calculation A

Fentanyl Grade Compute: Why a GB300 Rack Will Out-Price Blow by Weight in 2030 A GB300 NVL72 weighs roughly 1,580 kg fully populated and recent purchase orders put it at 5M per rack. That is 3,165/kg.

I'm about to buy a Mac Studio mainly for running LLMs locally and I'm stuck between two configs: M5 Ultra (30/64) with 96GB: 1.2 TB/s bandwidth, roughly 1.7x faster generation and much faster prefill

Hey everyone, Jovan from UkisAI here! Today, we are introducing Swift, a family of efficient reasoning LLMs based on Qwen, trained by penalizing tokens related to pathological overthinking patterns an

HIP : optimize IQ2/IQ3 (__vsub4 __vcmpne4) using SWAR (#27962) HIP : use bit manipulation for __vcmpne4 HIP : use bit manipulation for __vsub4 Website: https://llama.app Attestations: https://github.c

hex-dma: introduce direct-mapped DMA cache that is better suited for HVX FA mask handling (#29282) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49404271

vulkan: hide internal symbols to prevent duplicate-dlopen state destruction (#29139) Since #28732 our internal symbols are exported. A duplicate copy dlopened and dlclosed by ggml_backend_load_all() t

sycl : support op get_rows_back, only support fp32/fp16 (#25266) resovle confict support gedt_rows_back, update the ops.md Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cp

sycl: extend MMVQ GLU fusion, add rms_norm+scale and ssm_conv+silu fusions (#28931) sycl : extend MMVQ GLU fusion to mixed quant types; add rms_norm+scale and ssm_conv+silu fusions fixing spacing issu

ggml-meta: resolve multi buffer views (#29266) ggml-meta: resolve multi buffer views add TODO to revisit if graph allocator gets refactored Website: https://llama.app Attestations: https://github.com/

vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415) vulkan: optimize IQ4_XS matmul kernels Assisted-by: OpenAI Codex vulkan: address IQ4_XS review nits drop the dead LOAD_VEC_A != 8 branch in the IQ4_X

model : support Gemma4 DSpark draft backbone (#29226) dspark: add Gemma 4 draft support Add GGUF conversion and runtime support for full-attention and SWA Gemma 4 DSpark drafts, including tied output

server: Dedup the draft HF model via dedup-cache-models (#27934) server: Dedup the draft HF model via dedup-cache-models Fixes #27846 server: avoid capturing structured binding in lambda Website: http

server: accept OpenAI video_url content type and data: video URIs (#27921) The OpenAI chat completions API specifies content part type 'video_url' with a {'url': ...} object, and clients typically sen

jinja : parse unary +/- before variables (#29244) jinja : parse unary +/- before variables Lexer already emits unary_operator for -n / +n, and runtime executes unary -. Parse them at multiplicative pr

server: fix token counting API crash on sleep (#29309) server: wake up sleeping server correctly server: wake up sleeping server correctly (local aliases removed) Website: https://llama.app Attestatio

CUDA: enable sparse-fa for dsv4 prefill (again) (#29298) CUDA: enable sparse-fa for dsv4 prefill (again) CUDA: unroll the query loop of the sparse mask scan The query loop of flash_attn_mask_to_sparse

tests: add -b/--backend option to test-llama-archs for testing a specific backend (#27372) tests: add backend option to test-llama-archs Update tests/test-llama-archs.cpp Co-authored-by: Johannes Gäßl

arXiv:2609.23907v1 Announce Type: new Abstract: Generative models of neural activity could help characterize tissue dynamics, compare experimental conditions, and simulate population activity for appl

vendor : update cpp-httplib to 0.57.1 (#29239) Signed-off-by: Adrien Gallouët adrien@gallouet.fr Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49092442 ma

hexagon: new HMX-optimized GATED_DELTA_NET (#29199) hex-gdn: start putting together HMX support for GDN hex-gdn: working hmx but not-pipelined and slow for now hex-gdn: re-write vtcm layout handling a

ci : publish snapdragon builds in release workflow (#29007) The snapdragon CI builds packages only to feed the QDC device tests, so Hexagon NPU binaries never reached the releases page. Build both tar

chat : Fix Muse Glimmer tool-call first parser error (#29242) Fix Muse Glimmer tool-call first parser error Add test to verify Condense patterns remove test for trigger patterns Website: https://llama

convert: add MiMo-V2.6 support (#29257) convert: add MiMo-V2.6 support Hoist the K3 mxfp4 conversion repack into base.py so it can be reused Remove decoder from mmproj convert Update conversion/mimo.p