768gb vram for less than the price of one RTX 6000

I have always posted about budget builds on here, and often asked how we are going to run the next big models. Often Plenty of downvotes too or folks telling me that it's not running if I'm getting 5tk/sec. But whatever, the hunger and desire to go big has always kept me on the edge and looking for deals. Here's my latest build, 12x64gb cmp170hx. For less than 1 RTX 6000 pro costs. I also have it connected with fiber to my other rig for RPC when I need more memory. I haven't been posting much since I built this rig, because it's now more fun to talk to my machine. I run GLM5.3, DSv4.1Flash, Qwen3.8Flash, Qwen3.8-2.4T, KimiK3 and MiniMaxM3. Performance is great, a single RTX 6000 or M3 Mac Studio wish they could. Inference with vllm or llama.cpp I look forward reading the replies how API usage is cheaper, or how it will take 52 light years to break even or the noise, or the electrical cost. NOT. There will be more opportunities in the future, keep looking for them and pounce on them when they come. up, the demand is going to be high for compute for a long time. https://preview.redd.it/dunixwu6caqh1.jpg?width=4080&format=pjpg&auto=webp&s=a76a56c53aa0b9206383dbebde679bf3690fd5d2 https://preview.redd.it/glpcbg5cbaqh1.jpg?width=3072&format=pjpg&auto=webp&s=462e9677bb1f3ada6431b61cf46e2d3705eccab9 submitted by /u/segmond [link] [comments]

Related

• Flyweight: open-source C++/CUDA engine for running MoE models bigger than your VRAM on one GPU + system RAM. First PyPI release, looking for contributors.
• M5 Ultra 96GB vs M5 Max 128GB — is 2x bandwidth worth losing 32GB of RAM, with Qwen3.8-Flash-Next dropping tomorrow?
• Running Qwen3.8-Flash-Next locally on a 12GB VRAM card
• I tried running a 1.56TB MoE model on a 6GB RTX 4050 Laptop, Here’s the result

Source: r/LocalLLaMA | 2026-09-18