Your model fits. Your settings don't. Three Ollama defaults multiply your memory before the model ever loads.

ollama ps settles this in one line, and almost nobody runs it before changing their quant. The PROCESSOR column reports exactly one of three things: 100% GPU - every layer on the GPU 100% CPU - the GPU is not being used at all something like 48%/52% CPU/GPU - partially offloaded If it is not 100% GPU, stop there. That is your answer, and the model is not what is wrong. A model at 90% GPU does not run at 90% speed, because every token waits on the slow layers. One spilled layer costs more throughput than a whole quantisation step down. Now the part that catches people, because all three of these are defaults rather than things anyone chose: 1. The context you think you have is 4096. Ollama's default context window is 4096 tokens, whatever the model card advertises. A 128k model gets 4096 unless you set OLLAMA_CONTEXT_LENGTH or /set parameter num_ctx. When your prompt exceeds the real window the front of it is dropped, and the front is where your system prompt lives. The model did not "ignore" your instruction and is not "getting dumber on long inputs". The instruction was never in the window. 2. Parallel requests multiply that number. Required memory scales by OLLAMA_NUM_PARALLEL x OLLAMA_CONTEXT_LENGTH. The docs' own worked example: a 2K context with 4 parallel requests becomes an 8K context, with the allocation to match. So raising num_ctx 4x while parallelism is above 1 does not cost you 4x. It costs 4x times the parallel count, and that is usually the step that moves you from "fits" to "spilling". 3. You may be holding three models at once. OLLAMA_MAX_LOADED_MODELS defaults to 3 x the number of GPUs (or 3 for CPU inference), and each model stays resident for 5 minutes after its last use. That is the answer to "it fit this morning and spills this afternoon" - something you tried an hour ago is still sitting in VRAM. ollama stop evicts it immediately. The fix order that actually works, cheapest first: ollama ps. Confirm you are really spilling before you change anything. ollama stop whatever you are not using, and check nothing else holds VRAM. A browser with hardware acceleration is a frequent culprit. Drop the context to what you actually need. Most local chat never needs what got set. Quantise the KV cache. OLLAMA_KV_CACHE_TYPE=q8_0 is roughly half the memory of the f16 default at very little quality cost, q4_0 roughly a quarter with a more noticeable one. Needs Flash Attention enabled. Only then drop the model quant. Most people run that list backwards. They give up real model quality first, and still spill, because they never touched the thing that was actually eating the card. Every number above is out of Ollama's own FAQ, re-read today rather than recalled - these defaults have changed before and will again, so check yours rather than trusting a post. If the sizing side is useful, weights versus context memory per model and what the quant suffixes actually mean is written up here: https://noizz.io/local-ai submitted by /u/blossend [link] [comments]

Source: r/ollama | 2026-08-26