DeepSeek-V4-Flash-0731 UD-IQ3_XXS about 11t/s on 1x 7900 XTX 24GB + 3x MI60 32GB + 128GB DDR4

Hello, Also I want to join the hype of posting token specs. CPU: 2x Intel Xeon CPU E5-2650 v4 @ 2.20GHz RAM: 2x 4 Channel 2400MHz DDR4 GPU: 1x AMD Radeon 7900 XTX 24GB 3x AMD Instinct MI60 32GB Strange GPU combination, right? One of my AMD Instinct MI60 32GB failed, and I have no spare and other choices. Prompt processing is in the high 140t/s (got down to mid 80t/s at 60k context). Inference is a about 11t/s. llama.cpp command is not optimized. llama.cpp logs: 38.32.848.664 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 2048, progress = 0.03, t = 14.48 s / 141.44 tokens per second 38.47.371.961 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 4096, progress = 0.06, t = 29.00 s / 141.23 tokens per second 39.05.717.960 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 6144, progress = 0.09, t = 47.35 s / 129.76 tokens per second .. 51.08.617.986 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 65536, progress = 0.98, t = 770.25 s / 85.08 tokens per second 51.27.241.174 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 66682, progress = 0.99, t = 788.87 s / 84.53 tokens per second 51.34.905.170 I slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 67194, progress = 1.00, t = 796.54 s / 84.36 tokens per second 51.43.631.476 I slot print_timing: id 0 | task 0 | n_decoded = 100, tg = 11.96 t/s, tg_3s = 11.96 t/s 51.46.703.156 I slot print_timing: id 0 | task 0 | n_decoded = 135, tg = 11.81 t/s, tg_3s = 11.39 t/s 51.49.759.994 I slot print_timing: id 0 | task 0 | n_decoded = 171, tg = 11.80 t/s, tg_3s = 11.78 t/s .. 52.11.051.549 I slot print_timing: id 0 | task 0 | n_decoded = 427, tg = 11.93 t/s, tg_3s = 11.84 t/s 52.14.103.819 I slot print_timing: id 0 | task 0 | n_decoded = 464, tg = 11.95 t/s, tg_3s = 12.12 t/s 52.17.143.301 I slot print_timing: id 0 | task 0 | n_decoded = 501, tg = 11.97 t/s, tg_3s = 12.17 t/s 52.19.252.023 I slot print_timing: id 0 | task 0 | prompt eval time = 796902.50 ms / 67198 tokens ( 11.86 ms per token, 84.32 tokens per second) 52.19.252.029 I slot print_timing: id 0 | task 0 | eval time = 43980.11 ms / 526 tokens ( 83.61 ms per token, 11.96 tokens per second) llama.cpp version: 10223 (11924d4c1) llama.cpp backend: ROCm 7.2.4 llama.cpp command line: GGML_CUDA_P2P=1 llama-server -m DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf --temp 1.0--top-p 0.95--min-p 0.00 -fa 1 -c 1048576 -np 1 --chat-template-kwargs {"reasoning_effort":"max"} -lm none -mg 0 submitted by /u/Hyungsun [link] [comments]

Source: r/LocalLLaMA | 2026-08-02

Related