Best Ollama models/settings for an 8GB VPS (CPU only, ARM)? Running into memory & looping issues.

This Reddit thread discusses running Ollama on a resource-constrained 8GB CPU-only ARM VPS, addressing common challenges such as out-of-memory errors and model response looping. For purely CPU-only setups with 8GB system RAM, it is generally recommended to stick to models under 4B parameters , and using Q4_K_M quantization is considered the sweet spot, as it preserves most of a model's capability while dramatically shrinking memory requirements with minimal quality loss. Key configuration tips likely covered include setting environment variables such as OLLAMA_NUM_PARALLEL=1, OLLAMA_MAX_LOADED_MODELS=1, and OLLAMA_FLASH_ATTENTION=1 , as well as reducing context window size, since larger context windows consume significantly more RAM.

Downloading an AI model just to hit an OOM error is the worst. 📉
Has anyone actually gotten a reliable local AI system running?
Need help setting up ollama.
My 2026 Ollama Setup Guide: What Actually Works Best for Daily Use on Consumer Hardware
Hallucination problem

Source: r/ollama | 2026-04-16

Related