Experience sharing: How do you use your local models and for what kind of tasks?

Here is my experience, which I would like to share with you and I also would like to hear your thoughts and valuable tips&tricks. Hardware: Mac Mini M4 (32GB Unified Memory) Model Server: Ollama Orchestration: My own custom harness: AmarinthUI . It’s held together by rubber bands, duct tape, and code co-written with Claude Sonnet and Gemma, but it works as my daily driver interface. My models and use cases: I use three categories of models for different tasks which of course have different speeds & "intelligence": Small models (up to 2b params) like gemma4:e2b for fast repetitive task where speed is important, and doesn't needs to be that smart, but it needs to follow the instructions correctly, and must have reliable tool calling. I use this model with certain API's to retrieve informations like (Weather, WebSearch, HomeAutomation control, ToDoList & GroceryList management, etc.) This small model can reliably input/output json/csv/yaml format, and it's fast enough to not to feel too laggy. (60-100 tok/sec). I use it for like 20% of the time Middle sized models (3b-4b active parameters) usually MoE(mixtures of experts) models like: gemma4:26b (MoE), qwen3.6:35b (MoE), Laguna XS 2.1 (MoE) which have a higher parameter count but only 3b-4b are active for each token processing. With these models I usually get around 25-35 tok/sec. I use these models as my daily drivers for general questions, chat and information extraction, summaries, RAG etc. These are my go to model (i use them for 70% of the time) Dense models like qwen3.6:27b or gemma4:31b, with speeds between 5-12 tok/sec(excrutiatingly slow, i know). I use this when i need the smarts, but speed is not important. i just give it a task and then walk away and come back after 10-20 minutes for the results. I use these models if i want to debug some code, or different scripting, or deep research, where I throw some documents at it and ask for a researching these documents. For these tasks i don't need speeds i need accuracy and the least risk of halucinations. I use this around 10% of the time. TL;DR: The 3-Tier Workflow: Tier Model Type Speed Best For... % of Usage Fast Small (<2B) 60-100 tok/s API calls, JSON output, Home Automation, Web Search. 20% Balanced MoE (3B-4B active) 25-35 tok/s General chat, Summaries, RAG, Daily Q&A. 70% Heavy Dense (27B+) 5-12 tok/s Deep research, complex coding, heavy document analysis. 10% What is your current setup? Are you finding that MoE models are becoming the standard for local setups, or do you prefer the reliability of dense models? submitted by /u/Redangel1984 [link] [comments]

Source: r/ollama | 2026-07-31