Model Releases
Tried running 748B DeepSeek-V4.1-Flash on a €2k PC
I bought a used PC with an RTX 3090 24 GB, 128 GB DDR4 and an Intel i9-9900X last year for under €2,000. With the release of DeepSeek-V4.1-Flash-1017 I decided to try running it on my machine. It is o
I bought a used PC with an RTX 3090 24 GB, 128 GB DDR4 and an Intel i9-9900X last year for under €2,000. With the release of DeepSeek-V4.1-Flash-1017 I decided to try running it on my machine. It is of course the smallest quant available (1-bit), but I was still happy to see it working at 7.2 tok/s. The answer has a physics error in it (wondering if someone can spot it), and I don't see much practical use for it. Qwen models actually work pretty well on my setup. Still, it felt really nice to see a 748B parameter model running on such hardware. Here are some technical details: Unsloth UD-IQ1_S GGUF, 233 GB on disk llama.cpp built from the open PR #30817, CUDA 12.8 The 110 GB Engram table isn't loaded up front. It stays on the NVMe and only the rows the model needs are read on demand, which is how it fits. llama-server -m DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00001-of-00005.gguf -ngl 99 --n-cpu-moe 36 -ot "per_layer_token_embd.weight=CPU" -lm mmap -lzm on -t 10 -c 16384 --jinja --temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0 --chat-template-kwargs '{"enable_thinking":false}' Now I only need to decide whether to keep the model or free up the disk space :) submitted by /u/Vladowski [link] [comments]
Related
- Döner Bench DeepSeek-V4-Flash IQ2_XS running on a single RTX 3090
- Ran DeepSeek V4.1 Flash 1017 on a single RTX 5090
- 3 experiments running dsv4-flash-0731 q4+ quants on 128GB RAM + ~60 GB VRAM (with a quite bad pcie infra) with an acceptable tgs and relatively acceptable pp speed
Source: r/LocalLLaMA | 2026-10-21