DeepSeek v4.1 Flash 1017 for Dwarfstar (ds4)

Hey y'all, just following up on my PR to get the new vision tower for 1017 working in ds4 with the quant I promised (the language model loads already, but vision needed my PR). I built to the exact same recipe antirez used for his original DS4.1 Q2: 2-bit routed experts with 8-bit or higher for everything else, combined with the original FP8 engram table. This worked well for the original DS4.1, and initial testing looks good here too. The imatrix is built from 1017, not the original checkpoint. The wired backbone of the model is just over 150GB, so it will stream from SSD on 128GB systems with nearly all of the routed experts held in the hot cache, allowing for decode speeds up to 20tps on an M5 Max. The ~200GB engram table is paged in from SSD on-demand, not held in RAM. This model is significantly revamped, with ~40% of the routed expert tensors changed, 30% of engrams, and ~50% of the vision tower's expert tensors altered. By all appearances, it's the new leading open-weights model, so I had to give it a try. If you are curious too, let me know what you think! submitted by /u/returnity [link] [comments]

Source: r/LocalLLaMA | 2026-10-18

Related