Model Releases
DeepSeek V4 Flash 0731 hits 82.7% on Terminal-Bench 2.1 in an independent public-harness run (445 trials)
Disclosure: I’m the author of Ante. DeepSeek recently reported an 82.7% score on Terminal-Bench 2.1 for DeepSeek V4 Flash 0731. Its evaluation used “DeepSeek Harness minimal mode,” which hasn’t been r
Disclosure: I’m the author of Ante. DeepSeek recently reported an 82.7% score on Terminal-Bench 2.1 for DeepSeek V4 Flash 0731. Its evaluation used “DeepSeek Harness minimal mode,” which hasn’t been released yet. We wanted to see whether the reported result could be independently matched using a public, downloadable harness. With Ante 0.preview.71, we got: 368 successful trials out of 445 82.7% accuracy (±1.79 SE) 89 Terminal-Bench 2.1 tasks 5 trials per task max reasoning effort no skills enabled deepseek/deepseek-v4-flash-0731 through OpenRouter The complete Harbor job is public. It includes the pinned configuration and all 445 trial records, with rewards, exceptions, durations, and token usage. Deep seek v4 seems to be sensitive to harness and this is probably useful data for anyone who is interested Sources: DeepSeek’s reported result: https://api-docs.deepseek.com/updates/ Public Harbor run and configuration: https://hub.harborframework.com/jobs/b2a14e4b-a422-45f2-832e-cf2eec5c8bff Ante benchmark page: https://antigma.ai/eval submitted by /u/Exciting-Camera3226 [link] [comments]
Related
- DeepSeek v4 Flash has a nice bump in Capability
- Harness showdown: Claude Code vs OpenCode vs Pi with DeepSeek V4 Flash
- any reasonably fast public benchmarks I should run quants of deepseek flash 0731 on?
Source: r/LocalLLaMA | 2026-08-09