Model Releases
Finally got Qwen 3.8 Next running on my v100 6gpu setup (TP2 PP3)
Hey everyone, After wrestling with hardware and engine issues for days, I finally got Qwen 3.8 Next running properly on my multi-GPU rig. Thought I’d share the setup journey, benchmarks, and thermal r
Hey everyone, After wrestling with hardware and engine issues for days, I finally got Qwen 3.8 Next running properly on my multi-GPU rig. Thought I’d share the setup journey, benchmarks, and thermal results for anyone trying something similar. Seeing all the ongoing memes on Reddit about multi-GPU setups turning into absolute space heaters and catching fire, I decided to run some rigorous thermal tests to see for myself. he Troubleshooting Odyssey PCIe Link Speed Issue: Right after installation, one of the cards dropped to PCIe Gen 1 x16. Spent about 8 hours over two days diagnosing and fixing it. Finding the Right Engine: Started with sglang-v100, but kept hitting continuous OOM crashes. Someone on Reddit previously suggested the pxa engine, but that threw errors as well. Eventually tried 1cat-vllm, spent some time tweaking it, and finally hit a stable run! Configuration: Running with TP2 PP3. Currently, speculative decoding is limited to speculative=1. Setting it to 2 throws an OOM due to memory constraints (might look into optimizing this later, but for now, it works). Context & Memory Stats Plaintext INFO: Available KV cache memory: 8.78 GiB INFO: GPU KV cache size: 531,288 tokens INFO: Maximum concurrency for 262,144 tokens per request: 2.03x Performance Benchmarks 1. Prompt Processing (Prefill) Input Length (Tokens) Speed (tok/s) 1,024 (1K) 1,389 2,048 (2K) 2,536 4,096 (4K) 3,210 8,192 (8K) 4,336 16,384 (16K) 4,679 32,768 (32K) 4,470 65,536 (64K) 3,820 131,072 (131K) 2,759 2. Text Generation (MTP Comparison) Output Length (Tokens) Base Speed (No MTP, tok/s) Optimized Speed (MTP Enabled, tok/s) 128 22.23 41.34 256 22.32 42.48 512 22.70 43.04 1024 22.86 43.28 2048 22.83 43.38 Average 22.59 42.70 MTP nearly doubles generation throughput across the board. Thermals & Acoustics People often meme about multi-GPU rigs turning into space heaters or jet engines, so I ran a thorough thermal/stress test: Stress Test: Ran gpu-burn continuously for 20 minutes. Thermal Equilibrium: Temperatures peaked at 65°C and stabilized right around 64°C. Fan Curve: Based on my fan control script, the fans were only running at around 76% at 64°C. The cards stay well under 65°C without even needing full blast. Pretty happy with how stable, cool, and quiet this system turned out. submitted by /u/Odd_Caterpillar_2994 [link] [comments]
Related
- Running Qwen 3.8 next on 16vram+32ram - A useful/fun post for the gpu poors
- I might have found the perfect config parameters for qwen 3.8 27b
- exllamav3 comfortably beats llama.cpp running CPU-offloaded Qwen-3.8-Flash-Next on my setup!
- vllm + p2p driver hack + qwen 3.8 27B vs llamacpp + qwen flash next ?
- How well do multiple GPUs scale for LLM inference? (Trying to understand the basics)
Source: r/LocalLLaMA | 2026-09-20