b10643

hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (#26501) hexagon: use non-host bufs by default and make the backend fully async hex-hb: remove optional hostbuf support and fix async copy hex-unary: relax supported unary check hex-bufs: use same get_alignment for host bufs snapdragon: bump android_platform to 34 hex-rows: super hacky get/set rows for q8_0 hex-get-rows: fix q8_0 hex-get-rows: supprot for f16 and cleanup for q8_0 hex-get-rows: generic macros and specialized thread funcs hex-get-rows: add DMA pipeline, vtcm_layout and kernel params hex-set-rows: fix q8_0 support, add dma and tracing hex-tests: override nmse threshold for HTP of Q8_0 quants hex-fa: add support for Q8_0 with inplace dequantizers hex-get-rows: simplify type dispatch hex-rows: simplify GET/SET_ROWS DMA pipeline hex-async: add events, set/get-tensor-async and rest of the async api support hex-repack: use slice instead of expert in repack functions hex-cpy: update event/async-cpy logging hex-set-rows: optimize smaller tensors hex-geglu: fix perf regression with larger tensors hex-get-rows: add missing header hex-set-rows: add missing header hex-bufs: ressurect GGML_HEXAGON_HOSTBUF but disable it by default hexagon: do not reject ops with non-heaxon buffers hex-get-rows: apply >=32 restriction only for q8_0 hex-res: bump vtcm acquire timeout to 10 seconds hex-bufs: add support for cloning buffers between sessions to speed up tensor copies hex-async: rework event recording and batch flushing and integrate with meta backend hex-bufs: improved handling of repacked tensors hex-repack: handle get_tensor_2d offsets hex-dev: add support for devices with multiple NPUs hex-sync: add support for sync tokens to synchronize npu devices for async splits hex-mmap: cleanup mmap calls and add a retry for robustness hex-sync: add failsafe if sync wait gets stuck hex-sync: use sync_seq to check for completed events hex-sync: rotate tokens for extra robustness hex-devs: add supprot for legacy device names for now hex-bufs: add support for auto-cloning buffers from diff sessions hex-fusion: simplify and optimize htp-opnode fusion handling hex-sync: override opnode name so that it shows up in the profiles hex-trace: update scripts to handle multiple devices hex-sync: bump the size of the opbatch queue and number of sync tokens hex-cpy-sync: do not explicitly flush opbatches in cpy_tensor_async and add support for cpy-dma hex-sync: add graph-flush threshold to avoid single op batches hex-sync: add sync_peer so that we can flush peers we depend on during cross-device ops hex-bufs: introduce tensor->extra and shadow_bufs for repacking hex-l2: flush tiny tensors inline hex-sync: use explicit l2flush for sync tokens hex-extra: track weight flags via tensor extra hex-fence: rename sync to fence hex-repack: proper handling of set-tensor-2d in the shadow_buf hex-trace: remove obsolete opstage mask that we used for profiling hex-env: remove obsolete use_hmx variable hexagon: new unified run.py and build.py and updated docs snapdragon: update run script to auto-escapt test-backend-op -p argument hex-scripts: fix trailing spaces hex-scripts: fix flake8 warnings snapdragon: cleanup dst lib/bin dirs before copying new build hex-ops: add support for allreduce hex-ar: improved allreduce with dma pipeline hex-ar: align macros hex-ar: consistent use of fence_seq hex-ar: add AR_SELECT env var to select ALLREDUCE kernel or fallback hex-ar: add proper synchronize handling for ALLREDUCE hex-opbatch: looks like we now just rely on backend.synchronise to flush the batches, no need to flush them by threshold hex-ar: bump block size to improve dma efficiency hex-ar: fused ALLREDUCE+ADD hex-ar: cleaner fence buffer management hex-ar: futher allreduce tweaking to remove race conditions hex-ar: add simple solver and remove non-dma kernels hex-ar: add row-broadcast to fuse with bias ADD hex-fence: pass seq numbers via op_params hex-ar: allow for both entry/exit seq for completing entry wait hex-ar: align macros hex-ar: do not refetch broadcast row hex-fusion: move all fusion into opbatch::add_op for consistency with ALLREDUCE and things hex-fusion: fix incorrect MUL_MAT reordering hex-mm: make fused 2x and 3x matmuls more generic hex-fusion: move tensor fusion tagging to graph_compute hexagon: make sure to copy tensor->extra by value hex-get-rows: fix offset calc with row-chunking hex-repack: get_tensor_2d fixes for non-zero offsets snapdragon: make profile/trace scripts more robust and donot mix stdout/stderr by default hex-devices: use legacy device nameing by default to ease the transition hex-devices: hardcode CDSP domain IDs for current devices for now hex-optrace: improve multi-NPU timestamp alignment and overall handling of cycle values hex-optrace: more robust handling of the fence events Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43283873 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (ROCm 7.14) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Android: Android arm64 (CPU) Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.3 DLLs Windows arm64 (CUDA 13) (preview) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 7.14) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI

Source: llama.cpp Releases | 2026-08-27