Model Releases
b11307
llama : preserve original batch order for speculative decoding layer inputs (#29019) llama: preserve original batch order for layer inputs Assisted-by: Codex tests: cover layer-input order across KV l
llama : preserve original batch order for speculative decoding layer inputs (#29019) llama: preserve original batch order for layer inputs Assisted-by: Codex tests: cover layer-input order across KV layouts Assisted-by: Codex tests: exercise layer-input ordering on CUDA devices Assisted-by: Codex llama: make layer input reordering compatible with tensor split Copy each microbatch tensor from offset zero and restore original row order after synchronization. Extend the layer-input regression to cover tensor split and repeated reads and decodes. Assisted-by: Codex llama: restore token order for unmasked NextN embeddings Use the original-token mapping for unmasked NextN rows, including when layer-input capture is disabled. Keep masked NextN rows on the logits output mapping and preserve offset-zero tensor copies. Extend the existing regression to cover NextN alone, combined layer capture, and masked outputs with repeated decodes and getters. Validation: all 256 CPU/CUDA/tensor configurations pass. Qwen3.8-27B Q4_K_M MTP completes MT-Bench at concurrency 16 before and after. Assisted-by: Codex ggml: fix WebGPU reservation and OpenVINO hidden-state capture Reserve WebGPU vector attention scratch across batch sizes and refresh reservations when NextN capture settings change. Preserve requested OpenVINO outputs, dynamic shapes, sequence counts, and current graph bindings. Extend existing WebGPU regression coverage and enable strict allocation checks. Assisted-by: Codex llama: defer regression test and backend fixes to follow-ups Keep this PR focused on restoring token order for layer inputs and unmasked NextN embeddings. Remove the added regression test, OpenVINO and WebGPU changes, and the separate NextN reservation change. Assisted-by: Codex llama: keep n_embd declaration in its original position Assisted-by: Codex llama : pass token count to layer input extraction Assisted-by: Codex llama : name original batch indices batch_idxs Assisted-by: Codex llama : name extracted embedding indices embd_batch_idxs Assisted-by: Codex llama : tag target embedding reordering Assisted-by: Codex llama : tag extraction and name the index capture flag Assisted-by: Codex Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51659063 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries Ubuntu x64 (ROCm 10.0) Ubuntu x64 (OpenVINO) Ubuntu x64 (SYCL FP32) Ubuntu x64 (SYCL FP16) Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Android: Android arm64 (CPU) Android arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide Windows: Windows x64 (CPU) Windows arm64 (CPU) Windows arm64 (OpenCL Adreno) Windows x64 (CUDA 12) - CUDA 12.4 DLLs Windows x64 (CUDA 13) - CUDA 13.4 DLLs Windows arm64 (CUDA 13) - CUDA 13.4 DLLs Windows x64 (Vulkan) Windows x64 (OpenVINO) Windows x64 (SYCL) Windows x64 (ROCm 10.0) openEuler: DISABLED openEuler x86 (310p) openEuler x86 (910b, ACL Graph) openEuler aarch64 (310p) openEuler aarch64 (910b, ACL Graph) UI: UI
Related
Source: llama.cpp Releases | 2026-10-01