Model Releases
Ctx size in Ollama Vs llama.cpp
So I know that Ollama is built on top of llama.cpp, I decided to run a model on both and applied settings on llama.cpp like flash attention and q8/q4 kv cache. On Ollama (using the Python package) I w
So I know that Ollama is built on top of llama.cpp, I decided to run a model on both and applied settings on llama.cpp like flash attention and q8/q4 kv cache. On Ollama (using the Python package) I was able to get 260k context using the num_ctx option, it would take a bit to load but it would eventually. On llama cpp, anything over 32k causes an OOM, why is that? How can I reproduce? submitted by /u/circumcised_hobbit [link] [comments]
Related
- Ollama full GPU offload:
LLAMA_ARG_THREADS=1drastically reduced CPU usage with no performance penalty - UPS Fuse Exploded While Running an LLM
Source: r/ollama | 2026-09-18