Local Ai
2×RTX 3090 + EPYC box running qwen3.8-flash-next at ~38 tok/s
What I have: - CPU: EPYC 7551 (32c/64T, Zen 1) - Board: Supermicro H11SSL-i (SP3), Rev 2.0 - RAM: 128 GB DDR4-2133 (all 8 channels full) - GPU: 2x RTX 3090 (48 GB total, PCIe 3.0) - 1500 W PSU What I
What I have: - CPU: EPYC 7551 (32c/64T, Zen 1) - Board: Supermicro H11SSL-i (SP3), Rev 2.0 - RAM: 128 GB DDR4-2133 (all 8 channels full) - GPU: 2x RTX 3090 (48 GB total, PCIe 3.0) - 1500 W PSU What I run: - Qwen3-Flash-Next (177B total / ~6B active MoE, IQ4_XS) on Ilama.cpp. Experts live in system RAM, hot ones cached in VRAM. Single stream = 38 tok/s. Two parallel requests drop to ~4 tok/s each. Budget: ~$800. Realistically that's either one more RTX 3090 or a CPU upgrade (a Zen 2 "Rome" EPYC drops into the same board). A new motherboard is out of budget i think for now. Which gives more inference speed for this setup - adding the 3rd 3090, or swapping to a faster/newer CPU? And would more/faster RAM matter here? Curious what people running similar rigs have actually measured. I am also interested in having multiple agents running at the same time, which currently slows it down heavily, so keeping the performance at multiple agents parallel would be a huge boost as well! submitted by /u/ludos1978 [link] [comments]
Related
- Qwen3.8-Flash-Next turns 4xR9700 into a local AI powerhouse! 120 t/s TG and 12k t/s PP single request with optimized vLLM
- Qwen3.8-Flash-Next at 170K context on a single 96 GB card. ~110 tok/s.
- Qwen3.8-Flash-Next NVFP4 2xDGX Spark config: 50t/s decode, 2,900t/s prefill
Source: r/LocalLLaMA | 2026-09-12