Model Releases
Deepseek v4 flash - 100-150 faster t/s in prefill/pp.
You have two choices here (in order of pref): Downgrade CUDA from 13.3 to 13.1 (skip 13.2 due to bugs) <- prefer this (thanks to u/fairydreaming for pointing this out) Use this vibed fork that works w
You have two choices here (in order of pref): Downgrade CUDA from 13.3 to 13.1 (skip 13.2 due to bugs) <- prefer this (thanks to u/fairydreaming for pointing this out) Use this vibed fork that works with CUDA 13.3 https://github.com/vektorprime/working_ds4_speed I was troubleshooting this yesterday with the nvidia profiler and some LLM help (https://www.reddit.com/r/LocalLLaMA/comments/1vcs7bl/ds4_flash_full_model_in_offload_600_ts_pp_and/) Here's some more info on #1 (quote from fairydreaming) "Downgrade your CUDA and recompile. Starting with 13.2 DeviceTopK is used for top-k instead of argsort, this turns PP rate to crap." In short, DS4 Flash is spending a lot of time on things other than matrix multiplication. submitted by /u/fragment_me [link] [comments]
Related
- DeepSeek-V4-Flash-0731 on Bosgame M5 with RTX PRO 6000 Max-Q eGPU
- DeepSeek-V4-Flash-0731-UD-Q3_K_XL 3x3090 test results
- DeepSeek V4 Flash 0731 IQ2_M benchmark for Dual 3060 and 96GB RAM ≈ 3.5 tok/s.
- DeepSeek-V4-Flash-0731: When Low is higher than High
- First Kimi K3 results on home lab ~ 4t/s
Source: r/LocalLLaMA | 2026-08-02