Improving an ollama model with rag and raft

Building a local AI agent hierarchy on a budget — RAG, quantization, and testing whether fact-grounding pushes small models toward real creativity I don't have a CS background, a GPU, or much of a budget — just a 13GB RAM machine and a lot of stubbornness. Over three repos I've been using already-established techniques (not inventing anything new) to see how far that combination can go: RAG pipeline — grounding a local model against a docs database to cut hallucination Low-RAM Ollama config — getting multiple models to coexist on constrained hardware without it falling over Agent-under-manager hierarchy (current) — five specialized small models, strictly serialized so they never compete for RAM at once, routed by a manager model The part I actually care about is baked into repo 3: pairing a creative-writing agent with a fact-checking verifier, to test whether grounding pushes small-model output toward genuine creativity instead of just pattern-recombination. It's inspired by the idea that human creativity might depend on dedicated cognitive/language structures — not just raw scale — since that architecture is largely absent even in our closest primate relatives. Early finding: the verifier could be prompt-tuned to correctly ignore figurative language (stopped flagging things like "she felt exhilarated" as unsupported), but it consistently missed a real technical error — calling an O(n²) slowdown "exponential." That's not a wording problem, it's a knowledge gap, which is exactly why the next step is grounding the verifier against a retrieval database instead of relying on the model's own recall. Repo 1: https://github.com/tomey265/lowram-ollama-raft-rag (Repos 2 and 3 are the subsequent repos named "ollama_rag_raft_two" and "agent-under-manager-hierarchy" in that order. Thank you for your attention to this post) submitted by /u/Broad_Psychology_954 [link] [comments]

Source: r/ollama | 2026-08-25