Model Releases
I tried three ways of getting Gemma and Qwen to update a notes file in Ollama
I ran a small test for Dory to see whether a local model could update one fact in a synthetic notes file without losing any of the others. I asked Gemma and Qwen to rewrite the file, return a patch, o
I ran a small test for Dory to see whether a local model could update one fact in a synthetic notes file without losing any of the others. I asked Gemma and Qwen to rewrite the file, return a patch, or append the change, with 20 trials per method at 20, 60 and 150 facts. At 150 facts, Gemma 4 12B QAT got 20/20 for each method, while Qwen3 8B got 7/20 for rewriting, 0/20 for patches and 17/20 for appending. The run used Ollama 0.34.1 on an M5 Max with 128 GB memory: gemma4:12b-it-qat (Q4_0, temperature 1.0) and qwen3:8b (Q4_K_M, temperature 0.6). Context was 8,192 tokens at 20/60 facts and 16,384 at 150, with output caps of 1,600/4,000/9,400. Thinking stayed at the defaults. The original run allowed one retry with scorer feedback. I later rescored the recorded replies for content rather than formatting; I didn't rerun the models. Their sizes, quantisation and temperatures differ, so I wouldn't take this as a general model ranking. Claude wrote the runner; the benchmark details include the settings, prompts, replies and scoring. submitted by /u/tjqscott [link] [comments]
Related
- Testing Qwen 3.8 27B running locally on a single 5090 via Ollama
- Which small model works best with Ollama on a Raspberry Pi 5?
- Hey all im on a m5 pro macbook pro and i am just trying to runollama run qwen2.5-coder:14b - now I could be doing something wrong but keep facing error 500 or eof so i tried a llama 3 1b model and i still get error 500. i have tried restarting and deleting ollama and so much more but nothing....
Source: r/ollama | 2026-10-10