Agentic harness for small models

Hello; I'm a semi-beginner at local AI. I've been experimenting with this tech for a while, and I still haven't found a proper harness that fits my models, hardware, and needs. My use case is pretty simple: web search, fetching, and browser use. Summarizing websites and having the agent explain stuff to me / Q&A. Potentially some agentic RAG (or whatever RAG-style implementation works best for my specs). Filesystem capabilities and a shell. MCP servers are a must (I use the GitHub, Hugging Face, Reddit, OpenRouter, etc. ones). A sandboxed JS/Python environment would be cool too. PDF reading is highly valuable to me, so that too. So far, I've tried the following: A large LM Studio plugins/MCP roster. It works, but it's not ideal. Hermes Agent. It injects a ridiculous amount of context and does a bunch of prompt processing and other sci-fi stuff that slows the workflow down way too much. My specs: RTX 5050 8 GB VRAM 16 GB DDR5 RAM 13th-gen i5 CPU My models (best I've found): Ornith 1 9B; Unsloth Q4 quant; Q4 KV cache; 65,536 ctx Gemma 4 E4B QAT; Unsloth Q4 quant; Q4 KV cache; 131,072 ctx Gemma 4 E2B QAT; Unsloth Q4 quant; Q8 KV cache or full precision; 131,072 ctx Ling 3.0 Tiny; using their custom llama.cpp fork; Q8 KV cache; Q4 quant; 131,072 ctx InternScience_Agents-A1-4B; Bartowski Q6 quant; Q8 KV cache; 131,072 ctx All running with optimal sampling/settings in LM Studio. Do you guys know of a lightweight harness that fills these needs? I want to finally settle this nonstop tinkering I'm doing so I can actually stick with something good and only switch models when better ones come out. submitted by /u/_lhz- [link] [comments]

Source: r/LocalLLaMA | 2026-08-15

Related