Web automation with Ollama on a 4 GB GPU: Qwen 3.5 4B plans, a heuristic clicks, and saved tasks replay with no model at all

I've been building ANCHOR, a web automation agent that runs entirely on Ollama. You describe a task in plain language ("Register Maria Silva, IT department, contractor, and save"), point it at a site, and it runs the task in the browser with Playwright. Everything runs on a GTX 1050 Ti with 4 GB: no cloud, no API keys. How it uses the models The LLM only plans. Qwen 3.5 (4B or 9B) turns the request into goals and steps, using the names of the elements on the page. A deterministic heuristic finds each element and refuses when it isn't sure; then the user picks, and the choice is remembered. The model never clicks anything directly. Small context, by design. The model gets a short summary of the page (numbered elements), not the HTML: a plan uses about 3,000 tokens and 2 model calls on average. Vision, from the same model. Qwen 3.5 has the vision capability in Ollama, so ANCHOR uses it as a last check when the other checks doubt that a step worked: at 640 px, a question about a screenshot takes about 5.5 s on the 4B and 9.5 s on the 9B, and got all 6 probe questions right. Model profiles are a small JSON file (ollama-small for the 4B, ollama-medium for the 9B), with temperature 0. Times on the 1050 Ti a full task with planning: about 30 s with the 4B, 50 s with the 9B; a saved automation, replayed without the model: about 3 s, including one run per row of a spreadsheet. What's in so far (0.1 to 0.3): the engine with user disambiguation and choice memory, the planner and agent loop with replanning, saved automations with parameters and spreadsheets, recovery when the site changes (with a reviewable, undoable record), checks of each action's effect, confirmation before sensitive actions, and defenses against prompt injection from page content. A benchmark: on 30 tasks with pages altered in 5 levels (renamed IDs, synonyms, restructured layouts, cookie banners, lookalike decoys and injected instructions), ANCHOR completed 82% (4B) and 84% (9B) of the runs, vs 53% for a fixed-selector script and 68–72% for an agent where the LLM controls everything. It claimed success without doing the task 2 and 1 times in 150 runs, vs 33 and 41 for the LLM-in-control agent. These are development numbers; a closed test set is waiting for the 1.0 run. What's next: uploads, downloads, new tabs and data extraction (0.4), then a local web interface with a "Run" button per saved automation (0.5). I'd love people to try it with their own models and hardware, and tell me what breaks. I'm especially curious how other small models do as planners, and how it runs on CPU only (I haven't measured that). Repo (source-available, PolyForm Strict): https://github.com/LucasDantas2701/ANCHOR Disclosure: this is my own project (a research project at my university in Brazil). I designed the system and the evaluation, and wrote the code with heavy help from an AI assistant (Claude). English isn't my first language, so I also used it to help write this post. submitted by /u/RepresentativeKey122 [link] [comments]

[[project-i-built-an-ai-agent-that-runs-entirely-on-cpu-with-a|[Project] I built an AI Agent that runs entirely on CPU with a 1.5B parameter model — here's what I learned]]
Optimizing an Ollama (Qwen:2.5) AI Agent: Fixing Search Aggregation, Context Bleed, and Query Extraction
I tested a fresh GitHub download → Ollama → first local coding-agent task (72 seconds, no cloud API)
Released v0.8.0 of my local-first AI Agent Automation platform
Running Ollama with Pi agent and Qwen 3.6

Source: r/ollama | 2026-10-09

Related