Which small model works best with Ollama on a Raspberry Pi 5?

I've been comparing and working on Gemma 4, Phi-4-Mini, and Qwen3-4B for local LLM use on an 8 GB Raspberry Pi 5. The results were indeed close, but each model has a different strength: Gemma 4 E2B: around 7.6 tok/s, with multimodal support and function calling. Phi-4-Mini: around 5–8 tok/s, with a focus on reasoning and code generation. Qwen3-4B: around 5–7 tok/s, with strong tool-calling and multilingual capabilities. My comparison also considers quantization, memory pressure, swap usage, and how the models fit into local RAG and agent workflows. One major takeaway for me was that a model being able to load is not the same as being comfortable to use interactively. Larger variants can become noticeably slower once swap is involved. I also wrote about it here: https://www.actian.com/blog/developer/phi-4-mini-vs-gemma-4-vs-qwen3-best-llm-for-raspberry-pi-5/ For people using Ollama on Raspberry Pi or other low-memory hardware, which matters most in practice: tokens per second, context size, tool calling, or response quality? submitted by /u/InsideDebt6345 [link] [comments]

Related

• I gave five different local LLMs a town. They invented Facebook and a duck-based credit bureau. (MIT, self-hosted, you don't play it — you watch it)
• I benchmarked Gemma4:e4b vs Gemma3:27B vs GPT-4o-mini vs Gemini 2.5 Flash on a Mac Mini M4 Pro 24gb — full results
• I built a thing that benchmarks Ollama models on your own hardware and ranks them
• I built a desktop benchmark for local Ollama models — TTFT, tok/s, VRAM, with a HuggingFace GGUF catalogue

Source: r/ollama | 2026-09-23