Search: “models”

rpc : fix pre-rdma macOS versions (#27815) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43962220 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silic

vendor : update cpp-httplib to 0.54.0 (#27919) vendor : update cpp-httplib to 0.54.0 vendor : update cpp-httplib to 0.54.0 and 0.54.1 Website: https://llama.app Attestations: https://github.com/ggml-o

metal : Add fa-vec tuning for M3 Pro (#27963) Related issue: #27668 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43966636 macOS/iOS: macOS Apple Silicon

metal: add fa-vec tunings for M3 Ultra (#27999) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43968762 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple

rpc: fix apple rdma error spew on teardown (#27908) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43970664 macOS/iOS: macOS Apple Silicon (arm64) macOS Ap

ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960) some backends (Metal, SYCL, WebGPU) require additional memory for fleeting data for certain ops, which is reflected in their ge

common: rename --tensor-read-lazy to --lazy-mode, add -lzm shorthand (#27969) Rename the --tensor-read-lazy CLI argument to --lazy-mode, to match the internal lazy_mode parameter, and add a -lzm short

hip : optimize Q2_0 dot-product path for gfx1201 (#26753) hip/gfx1201: optimize q2_0 vec_dot_q2_0_q8_1 with native amdgcn perm Broadened HIP's Q2_0 perm optimization Remove redundant HIP perm availabi

hip: tune rdna 3 mmq config (#26284) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44012223 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (ar

CUDA: use the fast mm_ids_helper path for any n_expert_used (#27978) The optimized path grouped warp lanes by token and required warp_size % n_expert_used == 0, with a single hardcoded exception paddi

llama: improve TENSOR_READ_LAZY handling (#27837) force lazy tensor on cpu if lazy is on llama: improve TENSOR_READ_LAZY handling Website: https://llama.app Attestations: https://github.com/ggml-org/l

vulkan: Change mul_mat_id to pad K rather than N (#27925) The N padding is needed for mul_mat, but not mul_mat_id. For mul_mat_id, we indirect the row index through a shared memory lookup table which

metal : add fa-vec tunings for M1 Max (#27932) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43859747 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple S

vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (#27526) vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs remove one more fastdiv W

sycl: make --fit respect --fit-target better (#27629) improve the --fit algorithm to take into account the actual peak required VRAM for a given context size on a SYCL backend. This includes both prop

metal : add remaining fa-vec tunings for M4 Pro (#27915) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43865923 macOS/iOS: macOS Apple Silicon (arm64) mac

metal : assert shared memory padding (#27951) metal : assert shared memory padding cont : add ref Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43867822 m

metal : add fa-vec tunings for M4 Pro (#27824) This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discussions/27668 to add support for additional Apple GP

metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (#27863) metal : add fa-vec tunings for M5 This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discuss

sycl: use TILE for quantized KV decode on BMG (#26689) Route quantized KV decode to TILE on Xe2 (BMG) only, keep VEC on other archs until validated there. Website: https://llama.app Attestations: http

metal : add fa-vec tunings for M4 (#27875) This adds fa_vec_tuned_table records for Apple M4 to ggml-metal-tuning.cpp. Includes F16, Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0. (M4, 10 GPU Cores) Co-authored-by

Vulkan: add hoisting support for row IDs and expert count in shaders (#26686) vulkan: add hoisting support for row IDs and expert count in shaders use hoisted row ids in coopmat2 vulkan: address revie

ggml : fix conv_transpose_2d for multiple batches (#26132) ggml : fix conv_transpose_2d for multiple batches ggml_compute_forward_conv_transpose_2d_impl only computed the first batch (ne[3] of the des

bench: add --tensor-read-lazy (#27881) bench: add --tensor-read-lazy rm the alias rename to LLAMA_LAZY_MODE_* Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestation

hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (#26501) hexagon: use non-host bufs by default and make the backend fully async hex-hb: remove optional hostbuf suppor

llama : add --n-cpu-ffn option (#26622) common : dedupe --n-cpu-moe / --spec-draft-n-cpu-moe override loops common : add --n-cpu-ffn to CPU-offload dense FFN weights of first N layers common : general

metal : fix memory leaks due to missing autoreleasepools (#27758) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43372015 macOS/iOS: macOS Apple Silicon (a

args: add --video-* CLI arguments (#24318) args: add --video-* CLI arguments gen docs nits add mtmd_helper_init_opt Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attes

spec: Add benchmark-only synthetic speculative acceptance options (#27711) Add benchmark-only synthetic speculative acceptance to llama-server and llama-cli Address review comments Address review comm

Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan Backend (#27453) vulkan: add LIGHTNING_INDEXER op vulkan: updated lightning_indexer.comp and ggml-vulkan.cpp with 128-lane dot-pr

quantize: cap working memory size to avoid loading big tensors onto RAM (#27795) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43468853 macOS/iOS: macOS A

opencl: add bin kernels kernel_gemm_moe_q4_0_q8_1_dp4a_bin, kernel_gemm_moe_mxfp4_q8_1_dp4a_bin (#27768) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/434

ci : bundle HIP runtime DLLs with Windows ROCm release (#26973) Copy amdhip64_7, amd_comgr and rocm_kpack next to the binaries so the correct HIP runtime loads over the driver's copy in System32. Fixe

ci : build only the ggml-hip backend for windows-rocm release (#27753) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43506547 macOS/iOS: macOS Apple Silic

server: add ctx-per-slot (--kv-unified-per-slot) (#24124) Add ctx-per-slot argument for unifid KV cache Swap out ctx fractions for ctx pool slots Formatting cleanup Remove ctx-pool-slots, make ctx-per

hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (#27798) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43526462 macOS/iOS: macO

ggml-hexagon: add HTP unary ops for ABS and LOG (#27786) Add HVX-accelerated implementations for GGML_OP_LOG and GGML_UNARY_OP_ABS on the HTP backend. Register HTP_OP_UNARY_ABS and HTP_OP_UNARY_LOG in

ggml-meta: propagate buffer usage and call init on the new tensors (#27586) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43044002 macOS/iOS: macOS Apple

ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization (#26647) metal: WIP chunked SSD SSM_SCAN kernels for multi-token prefill metal: drop scalar SSD path; MMA + sequential tail drop WIP ss

cuda: unblock mmq for MoE on sm_60 (#26264) cuda: unblock mmq for MoE on sm_60 cuda: duplicate mmq-config-pascal for dp4a and older cuda: reduce occupancy on non-dp4a pascal for Q2_K, Q4_K, Q5_K, Q6_K

ci: Clean up UI builds from releases (#27706) ci : inline UI version resolution into ui-build.yml ci : build UI once and reuse the artifact in release jobs Server jobs now extract the ui-build artifac

Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (#27216) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43166320 macOS/iOS: macOS Apple S

vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps (#27726) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43173045 mac

llama: add token ID tracking to KV cell (#27762) kv: track token id rm get_prev_tokens, move it to the main pr nits add get_prev_tokens Website: https://llama.app Attestations: https://github.com/ggml

sycl : mark tq2_0 as not supported (#27660) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42777546 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Sili

grammar : parse - in char classes as literal hyphen (#27591) grammar : accept '-' escape in character classes gbnf_escape_char_class() escapes '-' as '-' but parse_char() rejected that escape, so gene