Is it possible to run it with a combined memory setup: 16 GB VRAM + 64 GB RAM + SSD for offloading n-grams?

Hardware: rtx 5080 16 gb vram; 64 gb ram ddr5 6000hz; ssd with unlimited memory; ryzen 7 9800 x3d. OS: Windows 11 Software: I’d prefer llama.cpp, but it’s not a strict requirement; I’ll use whatever you suggest, as long as it works on Windows. My attempts to run it with llama.cpp: llama-server ^ -m "F:.lmstudiomodelsnslothQwen-NextQwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf" ^ -c 10000 ^ --n-gpu-layers 999 ^ -b 512 ^ -ub 512 ^ --fit off ^ --parallel 1 ^ --jinja ^ --flash-attn auto ^ --load-mode mmap ^ --no-host ^ --override-tensor "per_layer_token_embd.weight=CPU" and 2nd attemtp: llama-server ^ -m "F:.lmstudiomodelsAtomicChatQwen3.8-Flash-Next-GGUFQwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64-00001-of-00033.gguf" ^ -c 10000 ^ -ngl 99 ^ -b 512 ^ -ub 512 ^ --fit off ^ --parallel 1 ^ --jinja ^ -fa on and i got 6 t/sec, its just unusable submitted by /u/Additional-Ordinary2 [link] [comments]

Optimizing Qwen3.6 / Qwen3.8-27B on 16GB VRAM: Complete Benchmark Results and Setup Guide (~30-50tps at 32k to 72k context)
After pushing 1M+ tokens through Qwen 3.8 27B, here is my optimal llama.cpp config for 16GB VRAM (73k Context, Agentic Coding)
16 GB VRAM purgatory discussion thread
Qwen3.8-Flash-Next (UD-IQ4_XS) on 2x RTX 3060 + 7800X3D, from initial 36 tps prefill to 400 tps and other benchmarks (-sm tensor trap) + VRAM/RAM usage

Source: r/LocalLLaMA | 2026-08-29

Related