5090: Windows or Linux for Qwen3.8.27b

I've got a dedicated AI rig sitting here with a RTX 5090 and 96GB RAM and for the past few years have been using Windows 11 and primarily LM Studio, but have also used vLLM, llama.cpp and Ollama. With Qwen3.8.27b I want to get the most out of this model. I get the feeling from the community that LM Studio on Windows might not be the best way to go. If you had this rig sitting on your desk and wanted the best environment to host Qwen3.8-27b, what would you use for it? (Intel i9-14900K, RTX 5090, 96GB DDR5) Windows 11 + which inference server Linux + which OS (Ubuntu?) & inference server I'm happy to replace Windows 11 with Linux if that's the best way to get the maximum performance out of this model. I'm not tied to Win11 if this is holding me up. I don't use it for anything else. UPDATE: After testing different KV cache settings with qwen/Qwen3.8-27B-Q6_K.gguf on my 5090 (Windows, LM Studio -- the t/s as reported via LM Studio built-in chat) with default MTP (0-3, 0.75): Q8_0 @ 128K: 28.9/31.5 GB VRAM, 88–92 t/s Q4_0 @ 128K: 26.8 GB, ~87 t/s Q4_0 @ 256K: 30.3 GB, ~88 t/s Q5_0 @ 256K: 29.9 GB, ~75 t/s Q8_0 @ 256K: 31.2 GB, 73 t/s first run, then 52 t/s So for now, Q4_0 @ 256K looks like the sweet spot: full context, still ~88 t/s, and enough VRAM headroom that I'm not riding the absolute limit. submitted by /u/toolman10 [link] [comments]

Related

• Anyone managed to get Qwen 3.8 27B running smoothly on vLLM? Can't get rid of endless thinking
• llama.cpp Windows Manager
• Running Qwen 3.6 35B A3B-Q8_0 gguf on a cheap radeon 7600 at 18 token/s * update increased to 21 t/s
• Running Qwen 3.5 35B A3B-Q8_0 gguf on a cheap radeon 7600 at 18 token/s
• Ollama Qwen3.6:35b randomly stops outputting tokens

Source: r/LocalLLaMA | 2026-08-16