Qwen3.8 Flash Next llama.cpp config tuning

Hola all. Do you guys mind sharing your LLama.cpp config and system setup details for Qwen3.8 Flash Next? Model's quite big and tryining many combinations of llama.cpp options takes lots of time, so looking for other people setup details. I've attached my current config at the bottom, so if anyone sees something that could be improved please shout. My current best result: - PP within 130...200 tps (limited by cpu?) - TG within 14..22 tps (~15tps on average) Hardware: - Dual RTX 3090 (48GB VRAM) - 128GB DDR4 - Some old Xeon 40 core - Proxmox VM, pcie passthrough, numa binding to a single phys cpu Llama.cpp config: llama-server --port ${PORT} --model /nvme/gguf/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf --mmproj /nvme/gguf/mmproj-Qwen3.8-Flash-Next-F16.gguf --load-mode none --lazy-mode off --parallel 1 --ctx-size 131072 --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --fit off --temp 1.0 --min-p 0.0 --top-p 0.95 --top-k 20 --presence-penalty 0.0 --repeat-penalty 1.0 --batch-size 2048 --ubatch-size 512 --split-mode layer -ts 26,10 -ngl 99 -ncmoe 26 --no-mmproj-offload --override-tensor per_layer_token_embd=CPU --chat-template-kwargs '{"reasoning_effort":"xhigh"}' ngl, ncmoe, ts - manually adjusted to fit the model without crashing submitted by /u/ChopSticksPlease [link] [comments]

Source: r/LocalLLaMA | 2026-09-12