Local Ai
3k$ 128GB VRAM + 256GB RAM DDR4 Server
I finished my home inference server. First I tried Lenovo p620 workstation and while it’s a good value overall it pissed me off with a ton of proprietary Lenovo shit to deal with and I return it in th
I finished my home inference server. First I tried Lenovo p620 workstation and while it’s a good value overall it pissed me off with a ton of proprietary Lenovo shit to deal with and I return it in the end. Components: 4xV620 - 1400 256GB DDR4 RDIMM 2666 - 610 Huanandzhi D12D - 410 EPYC 7452 - 170 PSU ASRock 1600 - 220 SSD Samsung 970EVO 1tb - Already had Case//Fans//Misc ~ 200 Power consumption is no shit ofc on such machine: 700-900w prefill 500-600w decode on Qwen3.8-next-flash Autoround W4A16 What it can do - EDIT: Qwen3.8-next-flash Autoround W4A16 1.3k prefill and 70tg code/60tg prose on 128k+ context with MTP-2 on vllm fork. I was disappointed with this machine and qwen3.8-27b speeds at first. But since Qwen3.8 next running good on it - I’m satisfied. Hope in more optimizations in future. submitted by /u/Thin_Pollution8843 [link] [comments]
Related
- My only real use case for a local AI use is document management, how much VRAM do I realistically need for a good experience?
- GPU guide (GB per dollar, bandwidth)
- 2× Radeon R9700 for Local AI Was Choosing AMD Instead of NVIDIA a Mistake Without CUDA?
- MI25 for 80-100€ worth it?
Source: r/LocalLLaMA | 2026-09-13