Model Releases
ggml-cpu/ops: vectorize flash-attention V-cache F16 to F32 conversion by jinzihao · Pull Request #26947 · ggml-org/llama.cpp
Overview ggml_cpu_fp16_to_fp32 leverages hardware F16C intrinsics (AVX-512, AVX2, etc.), faster than the software-only ggml_fp16_to_fp32_row, bringing 17-31% gain in prompt processing rate for a small
Overview ggml_cpu_fp16_to_fp32 leverages hardware F16C intrinsics (AVX-512, AVX2, etc.), faster than the software-only ggml_fp16_to_fp32_row, bringing 17-31% gain in prompt processing rate for a smaller model like qwen3:4b. Wish the PR had few additional models(recent ones like Qwen3.5/3.6 & Gemma-4 models) with t/s stats. submitted by /u/pmttyji [link] [comments]
Related
- kv-cache : avoid kv cells copies by ggerganov · Pull Request #24277 · ggml-org/llama.cpp
- llama.cpp PR reports up to 169% faster quantized-KV decode at 118K context on Intel Battlemage from one SYCL kernel switch
Source: r/LocalLLaMA | 2026-08-13