Which local models handle structured extraction well? I moved my agent's memory into a folder (BYO model) and want to run it on Ollama instead of Sonnet

Setup, so the question makes sense: my coding agent's memory now lives in a folder of Markdown — entities, episodes, and procedures (workflows with a per-step success/fail record). Nothing needs an account. Two things need a model: turning a transcript into those files, and, after a failed run, asking what belief broke and writing the next version of the workflow. Everything else (search, recall, the gate that makes the agent confirm before running an untested workflow) is plain code, no network. pip install mengram-ai mengram local init ./memory --provider ollama --model llama3.1:8b mengram import claude-code --memory ./memory # or: mengram local add "text" mengram local map --memory ./memory --open Screenshot is the map it renders: three workflows I actually ran since, with the record on each step and "last success" dates. Where I want your experience: I ran the import on Claude Sonnet 5 first. 17 sessions took ~30 minutes because it thinks before every answer, and one session sat for 10 minutes and never came back. The extraction prompt is ~3.6k tokens plus up to ~4k tokens of transcript, and it must return one JSON object with nested arrays. On Ollama: which model in the 7–14B range returns valid nested JSON reliably for you? llama3.1:8b, qwen2.5:14b, gemma, something else? 8K context minimum — is anyone running 16K+ on an 8B without it falling apart on the long prompt? does format: json help or hurt the quality of what comes back, in your experience? I'll post numbers for whatever people recommend. Honest limits of the thing itself: search is word overlap (fine for a few hundred files), the map is a snapshot you re-run, and a couple of workflows came out twice under slightly different names — dedup threshold is on the list. submitted by /u/No_Advertising2536 [link] [comments]

Source: r/ollama | 2026-09-08