Model Releases
gfx906-llama-cpp: New PP/TG gains for MI50/MI60/Radeon VII/AMD GCN
Time for another update! We have been busy and managed to improve the gains substantially (mostly from exploring existing llama cpp PRs and adopting relevant things). Among other things the README.md
Time for another update! We have been busy and managed to improve the gains substantially (mostly from exploring existing llama cpp PRs and adopting relevant things). Among other things the README.md was also appended to provide a better overall picture of what’s in the fork, why and from whom. metric upstream t/s fork t/s gain prefill PP16384 332.5 ~410 +23% 120k deep fill 231.4 ~264 +14% TG 13.6 ~15.1 +11% (parity pre-mirror) context cannot fit 250k on 40 GB tight-fit machinery outputs - - bit-identical (sha + token-for-token) https://github.com/milpster/gfx906-llama-cpp/blob/master/README.md (Yes i made this with AI) submitted by /u/milpster [link] [comments]
Related
- Update: llama.cpp for Radeon VII / MI50 / MI60 — +14% PP, +9% long-context fill vs upstream + adaptive Flash Attention
- New: Llama.cpp adaptive speculation for faster inference
- PSA for anyone with multiple V620's or other gfx1030 cards having problems making llama.cpp tensor split work -- set '-ub 384' and -b to a multiple of that depending on number of GPUs
- Llama.cpp ROCm 7.2->7.14 upgrade, Radeon 780m iGPU benchmarks: ROCm vs Vulkan
Source: r/LocalLLaMA | 2026-09-05