Built a local memory engine for Ollama that runs in <1.2GB VRAM without vector DBs (Hillock v0.5)

Hey r/Ollama, If you run local models on mid-range hardware, you probably know how fast VRAM disappears when you try to set up document memory or local RAG pipelines. I built Hillock (https://github.com/roandejager/Hillock) as a lightweight alternative that pairs directly with your local Ollama setup. The main difference is that it doesn't use an LLM to parse documents into memory. It uses a fast CUDA tensor pipeline (GLiREL + MiniLM) to extract Subject-Predicate-Object triples into SQLite in ~5 seconds. Query similarity gating happens on the CPU in under 1ms using 10,000-dimensional hypervectors. Ollama is only called at the very end to stream the grounded answer, and only if the question actually cleared the gate against verified facts. If there's no evidence, it refuses immediately without wasting GPU tokens. In v0.5.0 I added: - A live /model command that queries your local Ollama tags API so you can list and switch models right inside the console. - Real-time token streaming across all chat modes. - A 1-click launcher (run.bat / run.sh) that sets up the venv and models automatically. The whole thing stays under 1.2 GB of VRAM on a GTX 1070, and also runs fine in CPU mode. GitHub: https://github.com/roandejager/Hillock submitted by /u/Equivalent-Flan-1590 [link] [comments]

Related

• are there memory limit settings that can be changed?
• Give your local Ollama models a personal knowledge bank (graph-based, not just vector search)

Source: r/ollama | 2026-08-23