Optimal Realistic Local AI for Most

So you’ve got a 3090 or maybe even a 5090? Or more likely a 4060 8GB Ti. You wanna try local AI, you don’t know what it can/can’t do. 1) Install the best model you can. If you have a 3090 or a 5090, that’s Qwen 27b. If it’s a 4060, it’s Qwen 35b-3a. Search this forum, 35b-3a can RIP on an 8GB card. 2) get an open router account. This is the “big brain” that will help you run your local AI. 3) download Hermes agent harness, set up an architect profile that connects to Qwen-3.7 or kimi-K3 or GLM-5.2 on OpenRouter. 4) set up Hermes profiles for coder, worker and browser-reviewer that connect to your local model, be it Qwen-27b or qwen-35b-3a 5) set up your SOUL.md for the architect profile (that is connected to a good model on OpenRouter) to make it VERY CLEAR that its role is to PLAN (this is where local agents can’t touch big models) and launch subagents using the Hermes delegate_task tool. It is the architect and the delegator. Tell it to send out subagents to scan the codebase. Tell it to make a phased implementation plan using subagents. Tell it to write no code, but to use the subagent coders. 6) launch your next Hermes session with: Hermes -p architect (if using the CLI). Execute some prompts and watch it send out sub-tasks for your local AI. 6) profit submitted by /u/fire_inabottle [link] [comments]

Related

• What is the best intelligence/stable model currently for a single GB10/DGX spark?
• 4x 3090, 96gb vram what Model to drive Hermes?
• 5700, 48GB RAM and a 3090 24Gb. Best OS and framework/model?
• I built a tool to actually test which weights matter before quantizing, instead of guessing (Qwen3.6-27B, 3 builds: Bedrock/Tightrope/Gambit)
• Pipeline parallelism in llama.cpp may be wasting your VRAM

Source: r/LocalLLaMA | 2026-07-31