I'm going crazy. Why does it use the CPU...

I don't know what's happening. So, when i use models in openwebui everything works, ollama uses the GPU fully, everything's perfect. Then, I use Hermes, and loads the model in VRAM (fully, no offload needed), but then it starts using 50% of the CPU and the GPU spikes for 1 second every 8, and obviously it's slow. Nothing is in RAM. Same thing in Qwen Code, I can't figure out why. Before it worked. The same models work in openwebui. Specs: Intel xeon e5 2680 V4 32 GB DDR4 2133 MHz Nvidia Tesla P100 16 GB OS: Truenas scale (ollama in docker, official app) Thanks in advance Logs from when it did it with Qwen Code: load_tensors: loading model tensors, this can take a while... (load_mode = mmap) load_tensors: offloading output layer to GPU load_tensors: offloading 47 repeating layers to GPU load_tensors: offloaded 49/49 layers to GPU load_tensors: CPU_Mapped model buffer size = 787.50 MiB load_tensors: CUDA0 model buffer size = 7024.41 MiB time=2026-08-16T17:37:17.185Z level=INFO source=llama_server.go:1348 msg=waiting for llama-server to become available status=llm server not responding time=2026-08-16T17:37:17.480Z level=INFO source=llama_server.go:1348 msg=waiting for llama-server to become available status=llm server loading model cmn common_init_: added logit bias = -inf cmn common_init_: added <|tool_response> logit bias = -inf cmn common_init_: added <turn|> logit bias = -inf llama_init_from_model: enabling flash_attn since it is required for quantized V cache llama_context: constructing llama_context llama_context: n_seq_max = 1 llama_context: n_ctx = 65536 llama_context: n_ctx_seq = 65536 llama_context: n_batch = 1024 llama_context: n_ubatch = 1024 llama_context: causal_attn = 1 llama_context: flash_attn = enabled llama_context: kv_unified = false llama_context: freq_base = 1000000.0 llama_context: freq_scale = 1 llama_context: n_rs_seq = 0 llama_context: n_outputs_max = 1 llama_context: n_outputs_max_per_seq = 1 llama_context: n_ctx_seq (65536) < n_ctx_train (262144) -- the full capacity of the model will not be utilized llama_context: CUDA_Host output buffer size = 1.00 MiB llama_kv_cache_iswa: creating non-SWA KV cache, size = 65536 cells llama_kv_cache: CUDA0 KV buffer size = 352.00 MiB llama_kv_cache: size = 352.00 MiB ( 65536 cells, 8 layers, 1/1 seqs), K (q5_0): 176.00 MiB, V (q5_0): 176.00 MiB llama_kv_cache: attn_rot_k = 1, n_embd_head_k_all = 512 llama_kv_cache: attn_rot_v = 1, n_embd_head_k_all = 512 llama_kv_cache_iswa: creating SWA KV cache, size = 2048 cells llama_kv_cache: CUDA0 KV buffer size = 220.00 MiB llama_kv_cache: size = 220.00 MiB ( 2048 cells, 40 layers, 1/1 seqs), K (q5_0): 110.00 MiB, V (q5_0): 110.00 MiB llama_kv_cache: attn_rot_k = 1, n_embd_head_k_all = 256 llama_kv_cache: attn_rot_v = 1, n_embd_head_k_all = 256 sched_reserve: reserving ... resolve_fused_ops: resolving fused Gated Delta Net support: resolve_fused_ops: fused Gated Delta Net (autoregressive) enabled resolve_fused_ops: fused Gated Delta Net (chunked) enabled resolve_fused_ops: resolving fused Lightning Indexer support: resolve_fused_ops: Lightning Indexer enabled resolve_fused_ops: resolving fused DeepSeek V4 HC support: resolve_fused_ops: fused DeepSeek V4 HC pre enabled resolve_fused_ops: fused DeepSeek V4 HC comb enabled resolve_fused_ops: fused DeepSeek V4 HC post enabled sched_reserve: CUDA0 compute buffer size = 212.32 MiB sched_reserve: CUDA_Host compute buffer size = 256.32 MiB sched_reserve: graph nodes = 2545 sched_reserve: graph splits = 98 sched_reserve: reserve took 218.65 ms, sched copies = 1 cmn common_init_: KV cache shifting is not supported for this context, disabling KV cache shifting cmn common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable) clip_model_loader: model name: clip_model_loader: description: clip_model_loader: GGUF version: 3 clip_model_loader: alignment: 32 clip_model_loader: n_tensors: 11 clip_model_loader: n_kv: 36 clip_model_loader: has vision encoder clip_model_loader: has audio encoder clip_ctx: CLIP using CUDA0 backend load_hparams: projector: gemma4uv load_hparams: n_embd: 3840 load_hparams: n_head: 0 load_hparams: n_ff: 0 load_hparams: n_layer: 0 load_hparams: ffn_op: gelu_quick load_hparams: projection_dim: 3840 --- vision hparams --- load_hparams: image_size: 224 load_hparams: patch_size: 48 load_hparams: has_llava_proj: 0 load_hparams: minicpmv_version: 0 load_hparams: n_merge: 1 load_hparams: n_wa_pattern: 0 load_hparams: image_min_pixels: 92160 load_hparams: image_max_pixels: 645120 load_hparams: model size: 167.00 MiB load_hparams: metadata size: 0.00 MiB get_dummy_batch: warmup with image size = 768 x 768 get_dummy_batch: warmup with image size = 768 x 768 reserve_compute_meta: CUDA0 compute buffer size = 13.53 MiB reserve_compute_meta: CPU compute buffer size = 6.75 MiB reserve_compute_meta: graph splits = 1, nodes = 21 warmup: flash attention is enabled clip_ctx: CLIP using CUDA0 backend load_hparams: projector: gemma4ua load_hparams: n_embd: 640 load_hparams: n_head: 0 load_hparams: n_ff: 0 load_hparams: n_layer: 0 load_hparams: ffn_op: gelu_quick load_hparams: projection_dim: 3840 --- audio hparams --- load_hparams: n_mel_bins: 640 load_hparams: proj_stack_factor: 0 load_hparams: audio_chunk_len: 0 load_hparams: audio_sample_rate: 16000 load_hparams: audio_n_fft: -1 load_hparams: audio_window_len: -1 load_hparams: audio_hop_len: -1 load_hparams: model size: 167.00 MiB load_hparams: metadata size: 0.00 MiB get_dummy_batch: warmup with audio size = 3000 get_dummy_batch: warmup with audio size = 3000 reserve_compute_meta: CUDA0 compute buffer size = 58.59 MiB reserve_compute_meta: CPU compute buffer size = 7.32 MiB reserve_compute_meta: graph splits = 1, nodes = 4 warmup: flash attention is enabled init_audio: audio input is in experimental stage and may have reduced quality: https://github.com/ggml-org/llama.cpp/discussions/13759 srv load_model: loaded multimodal model, '/root/.ollama/models/blobs/sha256-675ad6e68101ca9413ec806855c452362f0213f2dfc5800996b086fdb8119842' srv load_model: initializing, n_slots = 1, n_ctx_slot = 65536, kv_unified = 'false' spec common_specu: no implementations specified for speculative decoding slot load_model: id 0 | task -1 | new slot, n_ctx = 65536 srv load_model: prompt cache is disabled - use --cache-ram N to enable it srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391 srv load_model: context checkpoints enabled, max = 32, min spacing = 8192 srv init: --cache-idle-slots requires --cache-ram, disabling srv init: init: chat template, example_format: '<|im_start|>system You are a helpful assistant<|im_end|> <|im_start|>user Hello<|im_end|> <|im_start|>assistant Hi there<|im_end|> <|im_start|>user How are you?<|im_end|> <|im_start|>assistant ' srv init: init: chat template, thinking = 0 srv llama_server: model loaded srv llama_server: listening on http://127.0.0.1:45959 srv update_slots: all slots are idle time=2026-08-16T17:37:25.540Z level=INFO source=llama_server.go:1360 msg=llama-server started in 23.82 seconds time=2026-08-16T17:37:25.762Z level=INFO source=images.go:374 msg=template selection model=registry.ollama.ai/library/gemma4-hermes:latest selected=renderer_parser renderer=gemma4 parser=gemma4 go_template=null chat_template=[tools completion vision audio thinking] harmony=null renderer_parser=[completion vision audio tools thinking] time=2026-08-16T17:37:25.763Z level=INFO source=sched.go:728 msg=loaded runners count=1 time=2026-08-16T17:37:25.763Z level=INFO source=llama_server.go:1293 msg=waiting for llama-server to start responding time=2026-08-16T17:37:25.764Z level=INFO source=llama_server.go:1360 msg=llama-server started in 24.06 seconds srv server_strea: conv_id= (empty=1) slot get_availabl: id 0 | task -1 | - skipping, slot is empty slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1 slot launch_slot_: id 0 | task -1 | sampler chain: logits -> logit-bias -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> ?temp-ext -> dist slot launch_slot_: id 0 | task -1 | sampler params: repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000 dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64 top_k = 64, top_p = 1.000, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 1.000 mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900 slot launch_slot_: id 0 | task 0 | processing task, is_child = 0 slot operator(): id 0 | task 0 | new prompt, n_ctx_slot = 65536, n_keep = 4, task.n_tokens = 34116 slot operator(): id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end) slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 1024, progress = 0.03, t = 27.04 s / 37.87 tokens per second slot operator(): id 0 | task 0 | cached n_tokens = 1024, memory_seq_rm [1024, end) slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 2048, progress = 0.06, t = 77.52 s / 26.42 tokens per second slot operator(): id 0 | task 0 | cached n_tokens = 2048, memory_seq_rm [2048, end) slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 3072, progress = 0.09, t = 139.16 s / 22.08 tokens per second slot operator(): id 0 | task 0 | cached n_tokens = 3072, memory_seq_rm [3072, end) slot print_timing: id 0 | task 0 | prompt processing, n_tokens = 4096, progress = 0.12, t = 211.24 s / 19.39 tokens per second blablabla same thing again again again. submitted by /u/Dzhmelyk135 [link] [comments]

Source: r/ollama | 2026-08-16