DeepSeek V4 Flash 0731 (Q4) now reaches 1,328 tok/s prefill and ~29 tok/s decode on one RTX PRO 6000

I've been working on speeding up DeepSeek-V4-Flash-0731 in Krasis and have now got the long-prompt prefill quite a bit faster on a single RTX PRO 6000 96GB. These are timing-disabled internal Krasis results using INT4 experts. They aren't HTTP round-trip speeds: Prompt size Prompt Processing about 1K 152 tok/s 2,043 321 tok/s 8,623 906 tok/s 23,348 1,328 tok/s 62,403 1,204 tok/s Decode after the roughly 1K prompt was 29.4, 28.2 and 28.5 tok/s when generating 50, 100 and 250 tokens. After the 62K prompt it was 19.4 tok/s, as each new token has a lot more context to attend to. Krasis streams the model through limited VRAM for full-GPU prefill, then keeps the hottest experts in VRAM and serves the rest from system RAM during decode. In this configuration it kept 6,440 of 11,008 routed experts resident. No expert pruning occurred. Krasis v1.0.19 can be downloaded here: https://github.com/brontoguana/krasis There is still more to optimise, particularly the prefill speed I think could go higher but I think the speeds are already useful for coding agents which tend to send a lot of context with every request. If anyone tries it on similar hardware let me know how it goes. submitted by /u/mrstoatey [link] [comments]

Source: r/LocalLLaMA | 2026-08-04