Model Releases
Where are you running DeepSeek V4.1 Flash reliably? I can't get it to finish a single 2-hour benchmark run
I reallise this is not exactly a locally hostable model, at least definitly not on my hardware. But I figured some of you might have some insight on this. I run an agentic coding benchmark. The model
I reallise this is not exactly a locally hostable model, at least definitly not on my hardware. But I figured some of you might have some insight on this. I run an agentic coding benchmark. The model gets tools (bash, file editing, a headless browser to playtest with), can spawn sub-agents, and has 2 hours to build a small 3D browser game. GLM, qwen , etc all ran through the same harness on OpenRouter and finished in 27 to 110 minutes without transport error. DeepSeek V4.1 Flash has never finished. It ran out of time on all tasks I tried. I've resumed the racing game run 4 times, about 6+ hours of wall clock in total, and it still hit the limit every time. It's cheap ($8.34 for 1,153 turns and 143M input tokens), so I'd really like to make it work to evaluate it. Setup: deepseek/deepseek-v4.1-flash on OpenRouter, Chat Completions with tool calling, max_tokens 32768, default routing with no provider pinning, streaming with usage included. Twice in one hour, OpenRouter sent a 502 provider_unavailable from Together partway through a streamed reply. Once the stream has started, OpenRouter can't fall back to another provider, so that request is just dead. 32 replies came back with no text and no tool call. 7 of them used exactly 32,768 output tokens (about 3.5 minutes each). The other 25 ended with a finish_reason that wasn't stop, length or tool_calls. One of those took 423 seconds and ~10k output tokens. Calls are very slow. Before I switched to streaming, 7 requests across two runs hit my 600s client timeout. So, for people using DeepSeek for long agentic sessions, where do you run it? DeepSeek's own API (I havent had much luck there either) or something like OpenRouter with a pinned provider? I guess I could startup 8xh200 on runpod or something, but's thats really overkill to benchmark something. I guess, my general goal with this convo is figuring out where you benchmark any large open source LLM before deciding to self host. submitted by /u/sebnadeau [link] [comments]
Related
- Minimum VRAM GPU to run DeepSeek-V4-Flash-0731 Q4_K_XL at around 30 t/s ?
- Extremely slow DSpark draft model performance (1-2 t/s) with DeepSeek-V4-Flash on llama-server compared to MTP?
- [[benchmark-llamacpp-batchubatch-impacts-on-pp-and-tg|[Benchmark] llama.cpp batch/ubatch impacts on PP and TG]]
- Anyone running DeepSeek-V4-Flash-0731 on MI325X with vLLM? Mine is behaving completely broken
Source: r/LocalLLaMA | 2026-09-15