Model Releases
What to run at 128GB VRAM?
Long time lurker, but I'm finally upgrading to 128GB VRAM, and I'm trying to figure out what to run. I had been leaning towards Qwen3.8 Flash-Next at ~Q4, and I generally prefer to not run anything be
Long time lurker, but I'm finally upgrading to 128GB VRAM, and I'm trying to figure out what to run. I had been leaning towards Qwen3.8 Flash-Next at ~Q4, and I generally prefer to not run anything below Q4. But I feel like the reception to Flash-Next has been a bit "meh", so I'm considering GLM 5.3 at ~Q2 or Deepseek 4 Flash at Q2 or Q3. I'm sure I'll try all 3, but I'm really curious what people in the same boat have been doing? Edit: configuration is 2 X CMP 170 HXs (64GB each) + ~256 GB of DDR4 RAM. Spilling into RAM is basically not an option, except for the ngrams and caching submitted by /u/Madigan37 [link] [comments]
Related
- 4x 3090, 96gb vram what Model to drive Hermes?
- 3 experiments running dsv4-flash-0731 q4+ quants on 128GB RAM + ~60 GB VRAM (with a quite bad pcie infra) with an acceptable tgs and relatively acceptable pp speed
- Qwen3.8-Flash-Next: Time to Update Those Benchmarks
Source: r/LocalLLaMA | 2026-09-10