Ollama full GPU offload: `LLAMA_ARG_THREADS=1` drastically reduced CPU usage with no performance penalty

I noticed that a recent Ollama/llama.cpp setup was using a surprising amount of CPU despite the model being fully GPU-offloaded. My setup: Ryzen 9 9950X VM with 16 vCPUs 3× RTX 4060 Ti 16GB Qwen 3.8 27B OLLAMA_NUM_PARALLEL=1 By default, llama.cpp appeared to size its CPU thread pool based on the 16 available vCPUs, resulting in significant CPU usage during inference. I tested different values of LLAMA_ARG_THREADS: LLAMA_ARG_THREADS Typical CPU usage 1 ~100% 2 ~150% 6 ~350% The interesting part is that generation performance did not improve with additional CPU threads. If anything, using fewer threads was slightly faster in my tests. Generation generally remained in the ~26–28 tok/s range: 1 thread: ~26–27.5 tok/s across multiple runs 2 threads: up to 27.9 tok/s after warm-up 6 threads: 26.84 tok/s after warm-up There was no consistent performance advantage from adding CPU threads, despite the large increase in CPU usage. So, at least on my fully GPU-offloaded multi-GPU setup, additional llama.cpp CPU threads appear to consume substantially more CPU without providing a measurable inference benefit, and may even slightly hurt performance. I'm therefore staying with: LLAMA_ARG_THREADS=1 One unrelated but useful discovery while benchmarking: I had {{CURRENT_TIME}} in my Open WebUI system prompt. This was effectively preventing prompt/KV cache reuse because the prompt changed on every request. After removing it, repeating the same ~9.4k-token context went from roughly: ~14.4 seconds prompt evaluation (~652 tok/s) to ~0.31 seconds (~30,000 tok/s) on the next run So if you're using Open WebUI + Ollama, dynamic values near the beginning of your system prompt are also worth checking. I'd be interested to see results from other people running recent Ollama/llama.cpp with full GPU offload, especially multi-GPU setups. submitted by /u/Fade78 [link] [comments]

Source: r/ollama | 2026-08-23

Related