Anyone running DeepSeek-V4-Flash-0731 on MI325X with vLLM? Mine is behaving completely broken

Is anyone here successfully running DeepSeek-V4-Flash-0731 locally with vLLM, especially on AMD MI325X? My setup: GPU: 1x AMD Instinct MI325X Model: deepseek-ai/DeepSeek-V4-Flash-0731 vLLM: 0.26.0 ROCm image --tokenizer-mode deepseek_v4 --reasoning-parser deepseek_v4 --tool-call-parser deepseek_v4 --enable-auto-tool-choice Open Webui hermes agent DSpark speculative decoding with the recommended greedy draft mode FP8 KV cache Basically following the official vLLM/DeepSeek MI325X recipe as closely as possible The model loads and runs, but its actual behavior is extremely strange and honestly feels broken. For example, I asked it about the weather. It started out normally, but then proceeded to: repeatedly tell me it had used different weather/browser tools repeat basically the same forecast several times claim it had opened/rendered things that it apparently hadn't output DSML tool calls in plain text claim file/tool operations succeeded when the runtime reported that they failed I've also seen it do things like: call nonexistent tools correctly identify a tool name in its reasoning and then call a different/nonexistent tool successfully generate a PDF and immediately tell me it generated a PNG output literal tool-call-like markup instead of making a proper structured tool call lose track of very basic conversation context This doesn't feel like a normal model-quality issue. It feels more like something is wrong with the chat template / DeepSeek V4 encoding / reasoning / tool calling integration, or possibly something specific to the ROCm vLLM implementation. submitted by /u/Brunofcsampaio [link] [comments]

Source: r/LocalLLaMA | 2026-08-07