Llama.cpp ROCm 7.2->7.14 upgrade, Radeon 780m iGPU benchmarks: ROCm vs Vulkan

With all the new models released recently one important upgrade went unnoticed: Llama.cpp bumped ROCm from 7.2 to 7.14. I was waiting for that because in 7.14 support for gfx1103 (Radeon 780m) was introduced. Although it is supported, gfx1103 build target is not set in pre-build llama.cpp binaries, so I had to build it from sources. But that was not difficult compared to CUDA, just followed official instructions. Here are my numbers on Ryzen 7 260 CPU, 64Gb DDR5 5600, Ubuntu 26, llama-bench parameters: -ngl 99 --load-mode none -ctk q8_0 -ctv q8_0 -t 8 --flash-attn on --progress -r 2 -p 8192 -p 16384 -n 128 -n 256 Qwen 3.6 35B A3B Q8 Unsloth | model | test | ROCm t/s | Vulkan t/s | | --------------------- | -------: | --------------: | --------------: | | qwen35moe 35B.A3B Q8 | pp8192 | 311.44 ± 0.19 | 288.31 ± 2.00 | | qwen35moe 35B.A3B Q8 | pp16384 | 278.58 ± 0.58 | 260.65 ± 0.93 | | qwen35moe 35B.A3B Q8 | tg128 | 18.35 ± 0.03 | 21.14 ± 0.02 | | qwen35moe 35B.A3B Q8 | tg256 | 17.98 ± 0.58 | 21.09 ± 0.12 | Qwen 3.8 27B Q8 Unsloth | model | test | ROCm t/s | Vulkan t/s | | -------------- | -------: | -------------: | --------------: | | qwen35 27B Q8 | pp8192 | 97.48 ± 0.20 | 66.45 ± 0.20 | | qwen35 27B Q8 | pp16384 | 90.66 ± 0.10 | 63.39 ± 0.00 | | qwen35 27B Q8 | tg128 | 2.71 ± 0.08 | 2.85 ± 0.01 | | qwen35 27B Q8 | tg256 | 2.33 ± 0.00 | 2.18 ± 0.18 | So, it only makes sense to use ROCm for dense models as it gives +50% pp speed up compared to Vulkan. P.S. One important caveat: for Vulkan I used amdgpu.gttsize=49152 amd_iommu=off ttm.pages_limit=16777216 kernel parameters, while for ROCm I had to remove amdgpu.gttsize part because of frequent crashes. Probably this limit is not compatible with GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 environment variable recommended for ROCm. submitted by /u/MaximusSenior [link] [comments]

Source: r/LocalLLaMA | 2026-08-16

Related